How to read from one stream and write to several at once?

Question

Suppose I have a readable stream, e.g. request(URL). And I want to write its response on the disk via fs.createWriteStream() and piping with the request. But at the same time I want to calculate a checksum of the downloading data via crypto.createHash() stream.

readable -+-> calc checksum
          |
          +-> write to disk

And I want to do it on the fly, without buffering an entire response in memory.

It seems that I can implement it using oldschool on('data') hook. Pseudocode below:

const hashStream = crypto.createHash('sha256');
hashStream.on('error', cleanup);

const dst = fs.createWriteStream('...');
dst.on('error', cleanup);

request(...).on('data', (chunk) => {
    hashStream.write(chunk);
    dst.write(chunk);
}).on('end', () => {
    hashStream.end();
    const checksum = hashStream.read();
    if (checksum != '...') {
        cleanup();
    } else {
        dst.end();
    }
}).on('error', cleanup);

function cleanup() { /* cancel streams, erase file */ };

But such approach looks pretty awkward. I tried to use stream.Transform or stream.Writable to implement something like read | calc + echo | write but I'm stuck with the implementation.

Answer 1

Node.js readable streams have a .pipe method which works pretty much like the unix pipe-operator, except that you can stream js objects as well as just strings of some type.

Here's a link to the doc on pipe

An example of the use in your case could be something like:

const req = request(...);
req.pipe(dst);
req.pipe(hash);

Note that you still have to handle errors per stream as they're not propagated and the destinations are not closed if the readable errors.