streamsummary-stream

Stream-based implementation of the StreamSummary data structure described in this paper.

Pipe in your buffers/strings to get approximate top-K most frequent elements.

var StreamSummary = require('streamsummary-stream');
var ss = new StreamSummary(50);
 
//...
 
myDataSource.pipe(ss);
 
ss.on('finish', function() {
  console.log(ss.frequency('42'));
  console.log(ss.top());
});

Requires es6 `Map`

This module uses es6 Maps, so you probably need node.js >= 0.12 or io.js.

API

StreamSummary(size, streamOpts)

Construct a new writable StreamSummary to track the size most frequent elements (extends Stream.Writable).

size - the number of elements to track
streamOpts - the options to pass to the Stream constructor

StreamSummary.frequency(element)

Get the approximate frequency of element. Returns null if the element isn't in the top size elements.

element - the value in question

StreamSummary.top()

Get the top size most frequent elements in ascending order of frequency.

StreamSummary.export()

Export the StreamSummary data as an object. Exported object will look like:

{
  size: 42,
  numUsedBuckets: 40,
  trackedElements: {...},
  registers: [...]
}

StreamSummary.import(data)

Import a StreamSummary data object (expects same format as export() returns).

data - object containing StreamSummary data

StreamSummary.merge(ss)

Merge another StreamSummary with this one. Returns a new StreamSummary of size equal to the combined sizes of the two.

ss - another StreamSummary instance

streamsummary-stream