What's dmr-source
Dmr-Source provides unified methods to achieve stream of data sources, such as local / remote (http / ftp) ...
- Source
- FileSource
- HttpSource
- FtpSource
- MultiSource
- SftpSource
- HadoopSource
Usage
Get file from ftp server to local
const fs = new FtpSource({ host: "127.0.0.1", path: "/source-ftp-test.dict", port: 21,});const writer = new FileSource().createWritableStream({path: "/home/work/a.log"});fs.createReadableStream().pipe(writer);Copy big file by FileSource
const fs = new FileSource({encoding: "utf8"});const reader = fs.createReadableStream({path: "/home/work/a.log"});const writer =fs.createWritableStream((option) => { option.path = "/home/work/b.log"; return option;});reader.pipe(writer);Link readables to one readable stream
const stream1 = new HttpSource({host: "localhost", path: "/a.log"}).createReadableStream();const stream2 = new FileSource({host: "localhost", path: "/b.log"}).createReadableStream();const ms = new MultiSource().add(stream1).add(stream2).add(() => { return new FileSource({host: "localhost", path: "/c.log"}).createReadableStream();});ms.createReadableStream().on("data", (chunk) => console.log(chunk));Events
const stream = new HttpSource({host: "localhost", path: "/404.page", timeout: 6000}) .createReadableStream();stream.on("error", (err) => { console.log("should be 404 error", err);});stream.on("end", (err) => { console.log("never arrive end");});