Search code examples
perlmojoliciousmojo-useragent

Mojo::UserAgent non-blocking vs blocking performance


I have the following code:

my $ua = Mojo::UserAgent->new ();

my @ids = qw(id1 id2 id3);

foreach (@ids) {    
    my $input = $_;
    my $res = $ua->get('http://my_site/rest/id/'.$input.'.json' => sub {
        my ($ua, $res) = @_;
        print "$input =>" . $res->result->json('/net/id/desc'), "\n";
    });    
}

Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

Why when I run the above code (non-blocking) does it take about 6 seconds while when running the code as blocking, i.e. inside the loop something like:

 my $res = $ua->get('http://my_site/rest/id/'.$input.'.json');
 print "$input =>" . $res->result->json('/net/id/desc'), "\n";

without the latest line it takes about 1 second?

Why is the blocking code faster than the non-blocking code?


Solution

  • The first thing to check when things happened. I couldn't get the same delay. Remember to try each way several times to spot outliers where there's a network hiccup. Note that the second argument to the non-blocking sub is a transaction object, normally written as $tx, where the response object is normally written res:

    use Mojo::Util qw(steady_time);
    
    say "Begin: " . steady_time();
    END { say "End: " . steady_time() }
    
    my $ua = Mojo::UserAgent->new ();
    
    my @ids = qw(id1 id2 id3);
    
    foreach (@ids) {
        my $input = $_;
        my $res = $ua->get(
            $url =>
            sub {
                my ($ua, $tx) = @_;
                print "Fetched\n";
                }
            );
        }
    

    One possibility is that keep-alive is holding an open connection. What happens if you turn that off?

        my $res = $ua->get(
            $url =>
            { Connection => 'close' }
            sub {
                my ($ua, $tx) = @_;
                print "Fetched\n";
                }
            );
    

    Here's a version that uses promises, which you'll want to get used to as more Mojo stuff moves to it:

    use feature qw(signatures);
    no warnings qw(experimental::signatures);
    
    use Mojo::Promise;
    use Mojo::Util qw(steady_time);
    use Mojo::UserAgent;
    my $ua = Mojo::UserAgent->new;
    
    say "Begin: " . steady_time();
    END { say "End: " . steady_time() }
    
    my @ids = qw(id1 id2 id3);
    
    my @gets = map {
        $ua->get_p( 'http://www.perl.com' )->then(
            sub ( $tx ) { say "Fetched: " . steady_time() },
            sub { print "Error: @_" }
            );
        } @ids;
    
    Mojo::Promise->all( @gets )->wait;