Node.js spawn: Keep StdOut and StdErr in the original order

Question

Trying to run windows batch script from node.js v12.6.0 and capture its output in real time and in correct order.

But order of stdout and stderr is often (not always, but in 80% cases) mixed during the tests.

How to keep stdout and stderr in the original order?

---

This is javascript snippet I use for testing:

const spawn = require('child_process').spawn;

// Using 'inherit' to fix:
// "ERROR: Input redirection is not supported, exiting the process immediately".

const options = {
    stdio: [
        'inherit', // StdIn.
        'pipe',    // StdOut.
        'pipe'     // StdErr.
    ],
};

const child = spawn('exectest.cmd', options);

let mergedOut = '';

child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk) => {
    process.stdout.write(chunk);
    mergedOut += chunk;
});

child.stderr.setEncoding('utf8');
child.stderr.on('data', (chunk) => {
    process.stderr.write(chunk);
    mergedOut += chunk;
});

child.on('close', (code, signal) => {
    console.log('-'.repeat(30));
    console.log(mergedOut);
});

I have tried to redirect stderr to stdout with:

child.stderr.pipe(child.stdout);

And remove stderr listener (handle only stdout):

child.stderr.on

To avoid race conditions, but stderr is not displaying in console, nor been added to 'mergedOut'.

---

This is contents of the batch file I trying to run (exectest.cmd):

@Echo Off
ChCp 65001 >Nul

Echo 1 (stdout) ...
Echo 2 (stderr) ... 1>&2
Echo 3 (stderr) ... 1>&2
Echo 4 (stdout) ...
Echo 5 (stdout) ...
Echo 6 (stderr) ... 1>&2

---

Current output:

1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
6 (stderr) ...
4 (stdout) ...
5 (stdout) ...
------------------------------
1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
6 (stderr) ...
4 (stdout) ...
5 (stdout) ...

Expected output:

1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
4 (stdout) ...
5 (stdout) ...
6 (stderr) ...
------------------------------
1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
4 (stdout) ...
5 (stdout) ...
6 (stderr) ...

---

EDIT 1:

With:

    stdio: [
        'inherit', // StdIn.
        'inherit', // StdOut.
        'inherit'  // StdErr.
    ],

Output order is reliably correct, so node.js itself seems to 'know' how to do this properly.

But in this case, I do not know how to capture the output:

child.stdout

is 'null', and attempts to listen to process stdout of node.js itself:

process.stdout.on('data' ...)

in any configuration gives 'Error: read ENOTCONN'.

---

EDIT 2:

If merge streams and listen to it this way:

const mergedStream = child.stdout.wrap(child.stderr);

mergedStream.on('data' ...);

We have one listener to both stdout and stderr, but ordering is still broken.

---

EDIT 3:

Is is possible to keep the correct order if you want to capture output OR display it.

To display proper output without capturing it, see 'EDIT 1'.

To capture proper output without displaying it real-time, just use:

child.stdout.on('data', (chunk) => {
    mergedOut += chunk;
});

child.stderr.on('data', (chunk) => {
    mergedOut += chunk;
});

But as soon as you try to reference to 'process.stdout / process.stderr' inside of the:

child.stdout.on('data' ...)

callback, ordering will break.

For example:

child.stdout.on('data', (chunk) => {
    process.stdout; // <-- This line will break everything.
                    // You do not even need to .write() to it!
});

Or if you try:

child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);

to avoid referencing to 'process' inside of the 'data' callback, ordering will break too.

Answer 1

Without modifying executable file, acceptable approach can be (notice spawn options and command arguments):

const { spawn } = require('child_process');

const options = {
    shell: true,
    stdio: [
        'inherit', // StdIn.
        'pipe',    // StdOut.
        'pipe',    // StdErr.
    ],
};

const child = spawn('exectest.cmd', ['2>&1'], options);

let mergedOut = '';

child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk) => {
    process.stdout.write(chunk, (_err) => { });
    mergedOut += chunk;
});

child.on('close', (_code, _signal) => {
    console.log('-'.repeat(30));
    console.log(mergedOut);
});

However, it has some drawbacks:

1. It spawn additional console process instance.
2. If you want to use 'windowsHide: true' spawn options parameter to hide application with GUI, avoid starting process with 'shell: true' parameter, because only console window will be hidden, not the target application window.
3. You can not distinguish StdOut and StdErr. Depending on output format of the target application, solution to this problem may look like this:

const readline = require('readline');

// ...

const lineReader = readline.createInterface({
    input: child.stdout
});

lineReader.on('line', (line) => {
    if (line.includes('[ERROR]:')) {
        console.error(line); // StdErr.
    } else {
        console.log(line);   // StdOut.
    }
});