Does write() (without callback) preserve order in node.js write streams?

Question

I have a node.js program in which I use a stream to write information to a SFTP server. Something like this (simplified version):

var conn = new SSHClient();
process.nextTick(function (){      
   conn.on('ready', function () {
      conn.sftp(function (error, sftp) {
         var writeStream = sftp.createWriteStream(filename);
         ...
         writeStream.write(line1);
         writeStream.write(line2);
         writeStream.write(line3);
         ...
      });
    }).connect(...);
});

Note I'm not using the (optional) callback argument (described in the write() API specification) and I'm not sure if this may cause undesired behaviour (i.e. lines not writen in the following order: line1, line2, line3). In other words, I don't know if this alternative (more complex code and not sure if less efficient) should be used:

writeStream.write(line1, ..., function() {
   writeStream.write(line2, ..., function() {
      writeStream.write(line3);
   });
});

(or equivalent alternative using async series())

Empirically in my tests I have always get the file writen in the desired order (I mean, iirst line1, then line2 and finally line3). However, I don't now if this has happened just by chance or the above is the right way of using write().

I understand that writing in stream is in general asynchronous (as all I/O work should be) but I wonder if streams in node.js keep an internal buffer or similar that keeps data ordered, so each write() call doesn't return until the data has been put in this buffer.

Examples of usage of write() in real programs are very welcomed. Thanks!

Answer 1

Does write() (without callback) preserve order in node.js write streams?

Yes it does. It preserves order of your writes to that specific stream. All data you're writing goes through the stream buffer which serializes it.

but I wonder if streams in node.js keep an internal buffer or similar that keeps data ordered, so each write() call doesn't return until the data has been put in this buffer.

Yes, all data does go through a stream buffer. The .write() operation does not return until the data has been successfully copied into the buffer unless an error occurs.

Note, that if you are writing any significant amount of data, you may have to pay attention to flow control (often called back pressure) on the stream. It can back up and may tell you that you need to wait before writing more, but it does buffer your writes in the order you send them.

If the .write() operation returns false, then the stream is telling you that you need to wait for the drain event before writing any more. You can read about this issue in the node.js docs for .write() and in this article about backpressure.

Your code also needs to listen for the error event to detect any errors upon writing the stream. Because the writes are asynchronous, they may occur at some later time and are not necessarily reflected in either the return value from .write() or in the err parameter to the .write() callback. You have to listen for the error event to make sure you see errors on the stream.