Can I capture STDOUT write events from a process in perl?

I need (would like?) to spawn a slow process from a web app using a Minion queue.

The process - a GLPK solver - can run for a long time but generates progress output.

I'd like to capture that output as it happens and write it to somewhere (database? log file?) so that it can be played back to the user as a status update inside the web app.

Is that possible? I have no idea (hence no code).

I was exploring Capture::Tiny - the simplicity of it is nice but I can't tell if it can track write events upon writing.

Solution

A basic way is to use pipe open, where you open a pipe to a process that gets forked. Then the STDOUT from the child is piped to the filehandle in the parent, or the parent pipes to its STDIN.

use warnings;
use strict;

my @cmd = qw(ls -l .);  # your command

my $pid = open(my $fh, '-|', @cmd)   // die "Can't open pipe from @cmd: $!";

while (<$fh>) {
    print;
}

close $fh or die "Error closing pipe from @cmd: $!";

This way the parent receives child's STDOUT right as it is emitted.^†

There is a bit more that you can do with error checking, see the man page, close, and $? in perlvar. Also, install a handler for SIGPIPE, see perlipc and %SIG in perlvar.

There are modules that make it far easier to run and manage external commands and, in particular, check errors. However, Capture::Tiny and IPC::Run3 use files to transfer the external program's streams.

On the other hand, the IPC::Run gives you far more control and power.

To have code executed "... each time some data is read from the child" use a callback

use warnings;
use strict;

use IPC::Run qw(run);

my @cmd = ( 
    'perl', 
    '-le', 
    'STDOUT->autoflush(1); for (qw( abc def ghi )) { print; sleep 1; }'
);

run \@cmd, '>', sub { print $_[0] };

Once you use IPC::Run a lot more is possible, including better error interrogation, setting up pseudo tty for the process, etc. For example, using >pty> instead of > sets up a terminal-like environment so the external program that is run may turn back to line buffering and provide more timely output. If demands on how to manage the process grow more complex then work will be easier with the module.

Thanks to ikegami for comments, including the demo @cmd.

^† To demonstrate that the parent receives child's STDOUT as it is emitted use a command that emits output with delays. For example, instead of ls -l above use

my @cmd = (
    'perl', 
    '-le', 
    'STDOUT->autoflush(1); for (qw( abc def ghi )) { print; sleep 1; }'
);

This Perl one-liner prints words one second apart, and that is how they wind up on screen.