Clussh
Stream task execution, through ssh, to one or many host, one or many times, in parallel or in series.
Features:
- Distribute tasks upon many hosts
- Stream task from stdin
- Stream task outputs and progression stats to ldjson stream
- Support scaling, parallel execution and retry, network failure and host unavailability
Installation
npm install -g clussh
This will install three command-line cli:
- clussh - The main command line
- clussh-board - A streamable clussh progress board
- clussh-log - Pretty print clussh logs
Pre-requirement: NodeJS and its awesome universe
Usages & exemples
# Run "Hello $(hostname)" on the default worker ssh://yourusername@localhost clussh # Run given command on default worker clussh --cmd 'echo "Hello world"'clussh --cmd 'echo "Hello world"' --scale 10 # Teen times clussh --cmd 'echo "Hello world"' --scale 10 --concurrency 2 # Two in parallel # Run given bash script on default worker clussh ./install.sh # Run given bash script on to agents (with a concurrency of 2 for agent-b) # (NB: for ssh cooking & baking reasons, the shell script must ends with an exit statement) clussh -w ssh://user@agent-a.local -w ssh://user@agent-b.local?concurrency=2 ./runme.sh # Pipe task from line delimited json stream echo '{ "id": "task-a", "cmd": "echo hello; sleep 1" }{ "id": "task-b", "cmd": "echo hello; sleep 2", "scale": 10 }{ "id": "task-c", "script": "./runme.sh", args: ["--option", "value"] }{ "id": "task-d", "script": "./runme.sh", "worker": "ssh://only@on.host" }' | clussh -w ssh://user@agent-a -w ssh://user@agent-b # You can pipe infinite line delemited json stream (eg. event logs for a file wacher...) infinite-stream | clussh # Print human readable logs clussh | clussh-log # Print nice-looking progress board clussh | clussh-board # Print human readable logs clussh | clussh-log # Save logs for further exploration (and still display the board) clussh | tee output.log | clussh-boardtail -f output.log | clussh-log # Another console... tail -f output.log | clussh-board # Another console... tail -f output.log | ndjson-filter 'd.type === "fail"' | prettyldjson # Another console...
Command line usage and configuration
clussh [options] [script filepath] Options: --help Show help [boolean] --version Show version number [boolean] --retry, -r How many retry
Clussh can be configured with a .clusshrc
file (see https://www.npmjs.com/package/rc)
Above, one worker uri overide the default clussh configuration using url query string (?concurrency=2
for host agent-b.local
). This will result in two workers for agent-b.local
: one with the default concurrency (4) and another with a concurrency of 2.
Log message format
Clussh output line delemited json stream consumable by clussh-board
or clussh-log
or any other line delemited json tools (see prettyldjson for pretty print and ndjson-cli for map, reduce, filtering, etc.)
TODO: Fields documentation & messages types
API
const clussh = const clusshStream = clusshStream
License: MIT - Novadiscovery