Barfer
This module provides a set of NLP tools, using other modules, to find various things:
- Sentiment (English and Spanish) using trigrams and bigrams
- Emoji sentiment
- Intention
- Topics
- Context
- Discover language (defaults to a white-list of eng, spa, por, fra and ger)
Changes
* 1.2.4 - Improved stopwords and sentiment, general system speed improvement
* 1.2.0 - Added more default taggers
* 1.1.1 - Fixing some bugs
* 1.1.0 - Overhaul to the core, removed a couple of modules, simplified logic, and, optimized parsing
How to use
Start Barfer
const barfer = new Barfer( { lang: { whitelist: [ 'spa' ] // works best when focused in one single language for now... // whitelist: [ 'eng', 'spa' ] }, } );Additional configuration options
const conf = { // process data as Twitter data twitter: true, // enable morphing into ascii only characters. latinize: true, // define a very important target/topic target: 'some name', // add interesting topics for the tagger. interesting: [ 'some', 'other' ], // set this to an empty array if you want to surpass the white-list lang: { whitelist: [ 'eng', 'spa', 'por', 'fra', 'ger' ] }, };Implement a term
// this term is a default in Barfer barfer.addParameter( ( tokens ) => { const match = /^(car|bus|metro|train|plane|boat|taxi|bike|bicicle)\b/igm.exec(tokens.rest()); if ( match !== null ) { return { tag: 'vehicles', length: match[ 0 ].length, data: match[ 0 ].toLowerCase() }; } } ); Study
const data = barfer.study( twit, ( err, data ) => { // data is full of rich data! } );Output
{ str: 'rt @jonbershad: @realdonaldtrump fun. so you won\'t be giving us a date when you\'ll be discussing your massive conflicts of interest?', lang: 'eng', topics: [ { count: 135, length: 10, stem: 'discuss', text: 'discussing', weight: 9.64, action: true, topic: true }, { count: 134, length: 6, stem: 'give', text: 'giving', weight: 9.57, stopword: true, action: true, topic: true }, { count: 59, length: 9, stem: 'conflict', text: 'conflicts', weight: 4.21, topic: true, sentiment: -2, negative: true }, { count: 35, length: 7, stem: 'massiv', text: 'massive', weight: 2.5, topic: true }, { count: 34, length: 6, stem: 'youll', text: 'youll', weight: 2.42, stopword: true, topic: true }, { count: 29, length: 9, stem: 'interest', text: 'interest', weight: 2.07, stopword: true, topic: true, sentiment: 1, positive: true }, [length]: 6 ], tagger: { actions: { tag: 'actions', words: { giving: { text: 'giving', count: 1, action: true }, discussing: { text: 'discussing', count: 1, action: true } } }, topics: { tag: 'topics', words: { giving: { text: 'giving', data: { index: 0 } }, youll: { text: 'youll', data: { index: 0 } }, discussing: { text: 'discussing', data: { index: 0 } }, massive: { text: 'massive', data: { index: 0 } }, conflicts: { text: 'conflicts', data: { index: 1 } }, interest: { text: 'interest', data: { index: 2 } } } }, positive: { tag: 'positive', words: { interest: { text: 'interest', data: [ 'massive', 'conflicts', [length]: 2 ] } } }, negative: { tag: 'negative', words: { conflicts: { text: 'conflicts', data: [ 'massive', 'interest', [length]: 2 ] } } } }, rest: [ 'discussing', 'massive', [length]: 2 ], sentiment: { polarity: -1, positive: { score: 1, words: [ 'interest', [length]: 1 ] }, negative: { score: -2, words: [ 'conflicts', [length]: 1 ] } }, emojiSentiment: { polarity: 0, positive: { score: 0, emoji: [ [length]: 0 ] }, negative: { score: 0, emoji: [ [length]: 0 ] } }, twitter: { parsedAt: 1481743750671, mentions: [ 'jonbershad', 'realdonaldtrump', [length]: 2 ], hashtags: [ [length]: 0 ], cashtags: [ [length]: 0 ], replies: [ [length]: 0 ], urls: [ [length]: 0 ] }, wordMap: { '@jonbershad': { count: 5, length: 12, stem: '@jonbershad', text: '@jonbershad', weight: 0.35, rest: true, mention: true }, '@realdonaldtrump': { count: 5, length: 16, stem: '@realdonaldtrump', text: '@realdonaldtrump', weight: 0.35, rest: true, mention: true }, fun: { count: 1, length: 4, stem: 'fun', text: 'fun', weight: 0.07 }, you: { count: 1, length: 3, stem: 'you', text: 'you', weight: 0.07 }, wont: { count: 1, length: 5, stem: 'wont', text: 'wont', weight: 0.07, stopword: true }, giving: { count: 134, length: 6, stem: 'give', text: 'giving', weight: 9.57, stopword: true, action: true, topic: true }, date: { count: 1, length: 4, stem: 'date', text: 'date', weight: 0.07, stopword: true }, when: { count: 1, length: 4, stem: 'when', text: 'when', weight: 0.07, stopword: true }, youll: { count: 34, length: 6, stem: 'youll', text: 'youll', weight: 2.42, stopword: true, topic: true }, discussing: { count: 135, length: 10, stem: 'discuss', text: 'discussing', weight: 9.64, action: true, topic: true }, your: { count: 1, length: 4, stem: 'your', text: 'your', weight: 0.07, stopword: true }, massive: { count: 35, length: 7, stem: 'massiv', text: 'massive', weight: 2.5, topic: true }, conflicts: { count: 59, length: 9, stem: 'conflict', text: 'conflicts', weight: 4.21, topic: true, sentiment: -2, negative: true }, interest: { count: 29, length: 9, stem: 'interest', text: 'interest', weight: 2.07, stopword: true, topic: true, sentiment: 1, positive: true } } }Note
This is proof of concept still, but I'm working regularly in improving it.
See test/index.js for more example in how to use Barfer.
Tests
Run VERBOSE=true npm test to run tests and see all data output
And npm test just to run the tests.
License
See LICENSE for license info