POST request with multipart/form-data content type to download a file

Question

There's a webpage which includes a button inside the form. Clicking on that button makes a POST request and a CSV file gets downloaded.

I'm trying to automate the CSV downloading process using LWP::UserAgent.

I noticed from Chrome Developer Tools that the Content-Type is multipart/form-data; boundary=---WebKitFormBoundary....

Any idea how can I send the exact Request Payload which developer tools is showing?

I generally do the below for x-www-form-urlencoded content type. But I have no idea how to submit multipart form data.

my $ua = LWP::UserAgent->new;
$ua->cookie_jar({ file => "cookie.txt", autosave => 1});


my $request = HTTP::Request->new('POST', $url);

#copy the form_data from chrome developer tools
my $form_data = 'key=val&key2=val2';

#the form_data is too big (and is in parts) in case of multipart content type

$request->content($form_data);
$request->header('Content-Type' => "application/x-www-form-urlencoded");
#I'll have to use `multipart/form-data; boundary=---WebKitFormBoundary....`

#add some other headers like 'Origin', 'Host' and 'Referer' in similar manner
#...
#...

push @{ $ua->requests_redirectable }, 'POST';

my $response = $ua->request($request);

#Get the file

Answer 1

Take a look at the entry on POST in the documentation for HTTP::Request::Common

You can generate a multipart/form-data request (instead of application/x-www-form-urlencoded) by passing a pseudo header value of Content_Type => 'form-data'

It would look like this

my $response = $ua->post(
    $url,
    Content_Type => 'form-data',
    {
        ...;  # Hash of form key/value pairs
    }
);