Search code examples
multithreadingperlconcurrency

Asynchronously execute subprocesses in perl and capture their output


I need to asynchronously run two commands in a Perl script and then process their outputs after both of the commands have finished executing.

So far I've tried using the threads package with the IPC::Run3 package. But this doesnt' work. This example doesn't produce any output:

#!/usr/bin/env perl

use warnings;
use strict;

use threads;
use IPC::Run3;

my $out1 = "";

my $t1 = async {
    run3([qw(echo foo)], \undef, \$out1, \undef);
};

my $out2 = "";

my $t2 = async {
    run3([qw(echo bar)], \undef, \$out2, \undef);
};

my $out3 = "";
run3([qw(echo bar)], \undef, \$out3, \undef);

$t1->join();
$t2->join();


print("$out1\n");
print("$out2\n");
print("$out3\n");

Solution

  • IPC::Run3 appears to have issues with threads, but that doesn't appear to the case for the more capable IPC::Run.

    use threads;
    use threads::shared;
    
    use IPC::Run qw( run );
    
    my $out1 :shared = "";
    my $t1 = async {
        run( [qw( echo foo )], ">", \$out1 );
    };
    
    my $out2 :shared = "";
    my $t2 = async {
        run( [qw( echo bar )], ">", \$out2 );
    };
    
    my $out3;
    run( [qw( echo baz )], ">", \$out3 );
    
    $t1->join();
    $t2->join();
    

    Don't forget to make your shared variable shared.

    Atlernative:

    use threads;
    
    use IPC::Run qw( run );
    
    my $t1 = async {
        my $out = "";
        run( [qw( echo foo )], ">", \$out );
        $out
    };
    
    my $t2 = async {
        my $out = "";
        run( [qw( echo bar )], ">", \$out );
        $out
    };
    
    my $out3 = do {
       my $out = "";
       run( [qw( echo baz )], ">", \$out );
       $out
    };
    
    my $out1 = $t1->join();
    my $out2 = $t2->join();