I am using node.js to launch some shell operation that takes some time, for instance:
find / -name myfile
So I have
require('child_process').spawn('find', ['/', '-name', filename]);
As I want the child process to continue if the main process dies (let's assume it also writes the results to a file), I use the following options:
require('child_process').spawn('find', ['/', '-name', filename], {detached: true, stdio: 'ignore'});
However, this means that I can't receive any data on the process while it is running, at least not through stdout/stderr. My question is whether a solution requires the child process to write to some resource, and the main process to poll this resource for updates, or whether there is some workaround for piping the data in a way that doesn't create a dependency between the child process and the main process (maybe with mkfifo?).
I found a nice solution using Redis Pub/Sub: both processes connect to a Redis channel of the same name, the main process subscribes to this channel when it starts/restarts, and the spawned process posts messages of progress to this channel with an associated process ID. It does require running a Redis server, but in this case I was already using Redis. The reason I like this solution is that it doesn't require manually polling any data on the drive or in memory, and can be scaled to multiple spawned processes writing messages to the same Redis channel.