Search code examples
perllwplwp-useragent

Reuse LWP Useragent object with HTTP POST query in a for/while loop


I am using LWP Useragent to make multiple POST calls with basic Authorization, wherein POST URL parameters are read from a CSV file. Here is my code:

use strict;
use warnings;

use LWP::UserAgent;
use JSON 'from_json';
use MIME::Base64 'encode_base64';
use Data::Dumper;

my @assets;
my %data;
my $response;
my $csvfile = 'ScrappedData_Coins.csv';
my $dir = "CurrencyImages";

open (my $csv, '<', "$dir/$csvfile") || die "cant open";
foreach (<$csv>) {
   chomp;
   my @currencyfields = split(/\,/);
   push(@assets, \@currencyfields);
}
close $csv;

my $url = 'https://example.org/objects?';

my %options = (         
        "username" =>  'API KEY',
        "password" => '' );  # Password field is left blank
        
my $ua = LWP::UserAgent->new(keep_alive=>1);
$ua->agent("MyApp/0.1");
$ua->default_header(
                    Authorization => 'Basic '. encode_base64( $options{username} . ':' . $options{password} )
);
my $count =0;

foreach my $row (@cryptoassets) {
          
          $response = $ua->post(
                     $url,
                     Content_Type => 'multipart/form-data',
                     Content => {
                           'name'=>${$row}[1],
                           'lang' => 'en',
                           'description' => ${$row}[6],             
                           'parents[0][Objects][id]' => 42100,
                           'Objects[imageFiles][0]' =>[${$row}[4]],
                         }
                  );
         
         if ( $response->is_success ) {
             my $json = eval { from_json( $response->decoded_content ) };
             print Dumper $json;
         }
         else {
          $response->status_line;
          print $response;
         }
}
sleep(2);
}

Basically, I want to reuse the LWP object. For this, I am creating the LWP object, its headers, and response objects once with the option of keep_alive true, so that connection is kept open between server and client. However, the response from the server is not what I want to achieve. One parameter value ('parents[0][Objects][id]' => 42100) seems to not get passed to the server in HTTP POST calls. In fact, its behavior is random, sometimes the parentID object value is passed, and sometimes not, while all other param values are passing correctly. Is this a problem due to the reusing of the LWP agent object or is there some other problem? Because when I make a single HTTP POST call, all the param values are passed correctly, which is not the case when doing it in a loop. I want to make 50+ POST calls.


Solution

  • I found the solution myself. I was passing form parameter data (key/value pairs) using hashref to POST method. I changed it to arrayref and the problem was solved. I read how to pass data to POST method on CPAN page. Thus, reusing LWP object is not an issue as pointed out by @brian d foy.

    CPAN HTTP LWP::UserAgent API

    CPAN HTTP Request Common API