Child_process handling a STDOUT stream with carriage return (\r)

Question

I'm writing a simple(ish) application that allows an internal system at work to request a copy process (using rsync) from a remote server to another remote server instigated using a REST call.

I'm already familiar enough with the express framework, and have just started experimenting with the child_process library, and stumbled upon a small problem.

I am successfully starting rsync processes using node's childProcess.spawn(), my issue is that rsync outputs its progress line-buffered with a carriage return (\r) instead of an newline (\n). As a result the STDOUT event, process.stdout.on('data', {}) is only called once before saying it's setting up the transfer, and then after the copy has finished, as STDOUT data isn't flushed on a carriage return, as the progress updates, only a newline, when the job finishes.

There is a switch in the latest version of rsync (3.1.0) to change the output buffer endings to \n instead of \r but unfortunately, the company I work at will not be adopting this version for a long time.

I'm spawning and reading the child_process in the usual way....

var doCopy = function (onFinish) {
    var process = childProcess.spawn('ssh', [
        source.user + "@" + source.host,
        "rsync",
        "-avz",
        "--progress",
        source.path + source.file,
        "-e ssh",
        dest.user + "@" + dest.host + ":" + dest.path
    ]);

    process.on('error', function (error) {
        console.log("ERR: " + error.code);
    })

    process.stdout.on('data', function (data) {
        console.log("OUT: " + data);
    });

    process.stderr.on('data', function (data) {
        console.log("ERR: " + data);
    });

    process.on('close', function (code) {
        console.log("FIN: " + code);
        if(onFinish){
            onFinish(code);
        }
    });
}

..and the console output is....

OUT: building file list ... 
OUT: 
1 file to consider

OUT: test4.mp4

         32,768   0%    0.00kB/s    0:00:00  
    169,738,240  32%  161.84MB/s    0:00:02  
    338,165,760  64%  161.32MB/s    0:00:01  
    504,692,736  96%  160.53MB/s    0:00:00  
    524,288,000 100%  160.35MB/s    0:00:03 (xfr#1, to-chk=0/1)

OUT: 
sent 509,959 bytes  received 46 bytes  113,334.44 bytes/sec
total size is 524,288,000  speedup is 1,028.01

FIN: 0

So you can see that the stdout.on('data,) is only called when rsync outputs a new line (where there is an 'OUT:').

My question is, can I change this? Maybe put the stream through a transform to flush when a \r occurs? Then I can regex the line and offer a progress update back again.

Failing this, I suppose my only other option would be to spawn another process to monitor the growing file?

Any help/advise is very much appreciated.

Answer 1

I've found this very good module that does exactly what I was trying to achieve. Allows me to delimit the stdout buffer by any character (in my case '\r') and trigger a new stdout event to handle the data. Like....

var splitter = process.stdout.pipe(StreamSplitter("\r"));

splitter.on('token', function (data) {
    console.log("OUT: " + data);
});

splitter.on('done', function (data) {
    console.log("DONE: " + data);
});