Writable Stream empties the file but do not write expected content in nodeJS

Question

This is my code to get an array of records and write them as CSV string one by one in a file. I cannot get the writable stream to write in the file. It just erases whatever is in the file and does not write the new data in.

const writeStream = fs.createWriteStream(process.env.SAVED_FILE);
const transformStream = new Stream.Transform({
    writableObjectMode: true,
    readableObjectMode: true,
})
transformStream._transform = (chunk, encoding, callback) => {
    console.log(chunk);
    transformStream.push("\"" + chunk.join("\",\"") + "\"");
    callback();
}
const readable = new Stream.Readable({objectMode: true})
    .pipe(transformStream)
    .pipe(writeStream);
let records = [["a", "b"], [1, 2], ["c", "d"], [8, 9]];
records.forEach(record => readable.push(record));
readable.push(null);

writeStream.on("error", function (err) {
    console.log("there is an error!")
});
writeStream.on("end", function () {
    console.log("reached the end of the stream!")
});

Any help appreciated on this basic example !

Answer 1

pipe() method returns the destination stream, i.e. the writable stream.

In your code,

const readable = new Stream.Readable({objectMode: true})
    .pipe(transformStream)
    .pipe(writeStream);

readable doesn't contains a reference to a readable stream like you expect. Instead, it refers to the last stream in the chain, i.e. writeStream and you cannot call push on a writable stream.

Change your code as shown below:

const readable = new stream.Readable({ objectMode: true });

readable
  .pipe(transformStream)
  .pipe(writeStream);