Search code examples
perlhttpcurllwp

lwp equivalent to curl with --data-raw $ sign


Curl command that works perfectly

curl 'https://someurl/graphql'\
-H 'authority: someurl' \
-H 'accept: application/json, text/plain, */*' \
-H 'accept-language: en-US,en;q=0.9,hi;q=0.8' \
-H 'authorization: Bearer PGTT78yyGLnRJGh...YJ8IF0ZeqzIXT' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary3M369HMcVXs0xNPZ' \
-H 'sec-ch-ua: "Chromium";v="106", "Google Chrome";v="106", "Not;A=Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36' \
--data-raw $'------WebKitFormBoundary3M369HMcVXs0xNPZ\r\nContent-Disposition: form-data; name="operations"\r\n\r\n{"operationName":"UploadImportFile","variables":{"organizationId":"43434","file":null},"query":"mutation UploadImportFile($organizationId: String\u0021, $file: Upload\u0021) {\\n  uploadImportFile(organizationId: $organizationId, file: $file) {\\n    uploadStatus\\n    uploadFileHeaders\\n    sampleData\\n    dataLinesCount\\n    filePath\\n    __typename\\n  }\\n}\\n"}\r\n------WebKitFormBoundary3M369HMcVXs0xNPZ\r\nContent-Disposition: form-data; name="map"\r\n\r\n{"1":["variables.file"]}\r\n------WebKitFormBoundary3M369HMcVXs0xNPZ\r\nContent-Disposition: form-data; name="1"; filename="sample_teacher_one_fresh_v2.csv"\r\nContent-Type: text/csv\r\n\r\n\r\n------WebKitFormBoundary3M369HMcVXs0xNPZ--\r\n' \
--compressed

Using https://corion.net/curl2lwp.psgi generated lwp code doesn't work and giving 500 error. After investigating more found this may be happening because of --data-raw with '$' symbol.

If I remove $ symbol from curl command results same 500 server error that is getting by generated perl lwp code. thanks in advance.

Generated Perl code:

#!perl
use strict;
use warnings;

use LWP::UserAgent;

my $ua = LWP::UserAgent->new( 'send_te' => '0' );
my $r  = HTTP::Request->new(
    'POST' => 'https://someurl/graphql',
    [
        'Accept'          => 'application/json, text/plain, /',
        'Accept-Encoding' => 'gzip, x-gzip, deflate, x-bzip2, bzip2',
        'Accept-Language' => 'en-US,en;q=0.9,hi;q=0.8',
        'Authorization'   => 'Bearer PGTT78yyGLnRJGh...YJ8IF0ZeqzIXT',
        'User-Agent' =>
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36',
        'Content-Length' => '782',
        'Content-Type' =>
'multipart/form-data; boundary=----WebKitFormBoundary3M369HMcVXs0xNPZ',
        'Authority' => 'someurl',
        'Sec-Ch-Ua' =>
          '"Chromium";v="106", "Google Chrome";v="106", "Not;A=Brand";v="99"',
        'Sec-Ch-Ua-Mobile'   => '?0',
        'Sec-Ch-Ua-Platform' => '"Linux"',
        'Sec-Fetch-Dest'     => 'empty',
        'Sec-Fetch-Mode'     => 'cors',
        'Sec-Fetch-Site'     => 'same-site'
    ],
"\x24------WebKitFormBoundary3M369HMcVXs0xNPZ\x5cr\x5cnContent-Disposition: form-data; name=\x22operations\x22\x5cr\x5cn\x5cr\x5cn{\x22operationName\x22:\x22UploadImportFile\x22,\x22variables\x22:{\x22organizationId\x22:\x2243434\x22,\x22file\x22:null},\x22query\x22:\x22mutation UploadImportFile(\x24organizationId: String\x5cu0021, \x24file: Upload\x5cu0021) {\x5cn uploadImportFile(organizationId: \x24organizationId, file: \x24file) {\x5cn uploadStatus\x5cn uploadFileHeaders\x5cn sampleData\x5cn dataLinesCount\x5cn filePath\x5cn __typename\x5cn }\x5cn}\x5cn\x22}\x5cr\x5cn------WebKitFormBoundary3M369HMcVXs0xNPZ\x5cr\x5cnContent-Disposition: form-data; name=\x22map\x22\x5cr\x5cn\x5cr\x5cn{\x221\x22:[\x22variables.file\x22]}\x5cr\x5cn------WebKitFormBoundary3M369HMcVXs0xNPZ\x5cr\x5cnContent-Disposition: form-data; name=\x221\x22; filename=\x22sample_teacher_one_fresh_v2.csv\x22\x5cr\x5cnContent-Type: text/csv\x5cr\x5cn\x5cr\x5cn\x5cr\x5cn------WebKitFormBoundary3M369HMcVXs0xNPZ--\x5cr\x5cn"
);
my $res = $ua->request( $r, );

Solution

  • The rough Perl equivalent of bash's

    $'...\r\n...'
    

    is

    "...\r\n..."
    

    or

    qq{...\r\n...}
    

    HOWEVER

    That's fine if you always want to send exactly the same thing. But what if the request changes and you want to build it?

    use Cpanel::JSON::XS      qw( encode_json );
    use HTTP::Request::Common qw( POST );
    
    my $query = <<'__EOS__';
    mutation UploadImportFile($organizationId: String!, $file: Upload!) {
      uploadImportFile(organizationId: $organizationId, file: $file) {
        uploadStatus
        uploadFileHeaders
        sampleData
        dataLinesCount
        filePath
        __typename
      }
    }
    __EOS__
    
    my $operations = {
       operationName => "UploadImportFile",
       variables => {
          organizationId => "43434",
          file => undef,
       },
       query => $query,
    };
    
    my $map = {
       1 => [
          "variables.file",
       ],
    };
    
    my $request = POST( 'https://someurl/graphql',
       Accept             => 'application/json, text/plain, /',
       Accept_Encoding    => 'gzip, x-gzip, deflate, x-bzip2, bzip2',
       Accept_Language    => 'en-US,en;q=0.9,hi;q=0.8',
       Authorization      => 'Bearer PGTT78yyGLnRJGh...YJ8IF0ZeqzIXT',
       User_Agent         => 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36',
       Authority          => 'someurl',
       Sec_Ch_Ua          => '"Chromium";v="106", "Google Chrome";v="106", "Not;A=Brand";v="99"',
       Sec_Ch_Ua_Mobile   => '?0',
       Sec_Ch_Ua_Platform => '"Linux"',
       Sec_Fetch_Dest     => 'empty',
       Sec_Fetch_Mode     => 'cors',
       Sec_Fetch_Site     => 'same-site',
    
       Content_Type => 'form-data',
       Content => [
          operations => encode_json( $operations ),
          map => encode_json( $map ),
          1 => [
             "sample_teacher_one_fresh_v2.csv",   # Sends the content of this file
             "sample_teacher_one_fresh_v2.csv",   # Name of file sent in request
             Content_Type => "text/csv",
          ],
       ],
    );
    
    my $response = $ua->request( $request );
    

    If you want to provide the CSV data from a variable instead of a file, replace the last parameter with

          1 => [
             undef,
             "sample_teacher_one_fresh_v2.csv",   # Name of file sent in request
             Content_Type => "text/csv",
             Content => '...',                    # CSV data to send
          ],
    

    Note that you don't have to use HTTP::Request::Common directly. The following is equivalent:

    my $response = $ua->post(
       ...  # Everything passed to POST above
    );