jsonlines-processor
JSON Lines streaming processor for CLI
A JSON Lines is called newline-delimited JSON and used for structured logs.
Install
$ npm install -g jsonlines-processor
How to Use
Create logic.js
exports { // filter if itemtarget === 'my_require' return item; else if item instanceof Array // Return multiple items item; } // Optionalexports { // sort return this} exports { // const anotherLogs = await this.readJSONLinesFile('./another.log') // const userMap = this.keyBy(anotherLogs, 'user.name') // await startServer()} exports { // await shutdownServer()}
Run command
$ gunzip -c application-json.log.gz | jlp logic.js > output_json.log
Helper function
Following utility methods can be called in process
, finalize
, before
or after
functions
sort(items, [key], [direct])
Arguments
items:Array
- The array to process[key]:String
- Target field name. item itself if not specified[direct]:String
- Ascending if not specified, else descending
Returns
Array
- the new array of sorted items
keyBy(items, key)
Arguments
items:Array
- The array to processkey:String
- The iteratee to transform key
Returns
Object
- the composed aggregate object.
sum(items, [key])
Arguments
items:Array
- The array to process[key]:String
- Target field name. item itself if not specified
Returns
Number
- the total value for each items
readJSONLinesFile(fileName)
Arguments
fileName:String
- JSON Lines file path
Returns
Array
- the new array of JSON object
readTSVFile(fileName)
Arguments
fileName:String
- TSV file path
Returns
Array
- Returns the new array of array item
Examples
example-json.log
logic1.js
Extracting the name and score of only item whose age is greater than 16, and sorting items by their score in descending order
exportsprocess = async { if age > 16 return name score } exports { return this}
Result
$ jlp logic1.js < example-json.log
{"name":"Ken",score:90}
{"name":"Taro",score:81}