Search code examples
phpxmlrestcurl

curl -F [email protected] in PHP

I use this cmd to successfully download result.csv

curl.exe https://example.com/rest -u "xx:yy" -F [email protected] > result.csv

I want to do this using PHP. But no avail... Help please.

This is my PHP:

$url = 'https://example.com/rest';
$xml = '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<request xmlns="http://example.com"><search path="report_date" value="2023-07-01T00:00:00+0000"/>'; //and so on..
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERPWD, 'xx:yy');
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, "xmlRequest=$xml");
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

$data = curl_exec($ch);
var_dump($data); //just to check if it has requested data (will save to file later)
if(curl_errno($ch))
    print curl_error($ch);
else
    curl_close($ch);

It does not have requested data. But it gives this

string(346) " 500 An error has occured. Please try again and if the problem persists, contact us. "

Solution

  • What you can say as a rule of thumb is that you look for each of the curl(1) command line options (+ arguments) and operands and find the appropriate curl_setopt($option, $value).

    So you can compare the command-line against the PHP code to locate potential issues that may cause the 500 error (ref). Otherwise, you can't find out what is wrong in the PHP code while the command-line works.

    In the following answer, I'll show how this is done step-by-step. And it has PHP code examples of how to upload a file both from the file-system and from a string with the file-contents.

    So let's take a look at what we've got here in that command-line:

    curl.exe https://example.com/rest -u "xx:yy" -F [email protected] > result.csv

    1. https://example.com/rest (URL)

      The URL syntax is protocol-dependent. You find a detailed description in RFC 3986. (ref)

    2. -u "xx:yy" (<user:password>)

      Specify the username and password to use for server authentication. (ref)

    3. -F [email protected] (<name=content>)

      For HTTP protocol family, this lets curl emulate a filled-in form in which a user has pressed the submit button. This causes curl to POST data using the Content-Type multi‐part/form-data according to RFC 2388.

      This enables uploading of binary files etc. To force the 'content' part to be a file, prefix the file name with an @ sign. To just get the content part from a file, prefix the file name with the symbol <. The difference between @ and < is then that @ makes a file get attached in the post as a file upload, while the < makes a text field and just get the contents for that text field from a file (ref)

    Then let's see which curl_setopt() operations there are in the PHP code:

    $ch = curl_init();
curl_setopt($ch, CURLOPT_USERPWD, 'xx:yy');
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, "xmlRequest=$xml");
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
    1. curl_setopt($ch, CURLOPT_USERPWD, 'xx:yy'); - OK, matches 2.) above.
    2. curl_setopt($ch, CURLOPT_URL,$url); - OK, matches 1.) above.
    3. curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); - OK, your config. (As you've written, you want to do this later.)
    4. curl_setopt($ch, CURLOPT_TIMEOUT, 10); - OK, your config. (Also good for faster testing.)
    5. curl_setopt($ch, CURLOPT_POST, true); - OK, mirrors the POST method matching 3.) above.
    6. curl_setopt($ch, CURLOPT_POSTFIELDS, "xmlRequest=$xml"); - NOT OK, does not match 3.) above. 3.) above does a file-upload, here you're setting a form string value. This is never a file-upload. Interesting, let's look into that.

    I can imagine this could trigger a 500 on the remote side, as they could have little error handling and the process just crashes, hence giving an 500 Internal Server Error and not providing information of what in detail is wrong with the request itself (400-499 HTTP status code range).

    Let's see how to turn this into a file-upload. There are two variants: From string or from pathname.

    PHP CURL File Upload from String

    You perhaps want to create a file-upload from the string so that the data of the file-upload is the string contents.

    If that is the case, this is how it works in PHP Curl tailored to your -F [email protected] curl(1) option and argument example:

    curl_setopt($ch, CURLOPT_POSTFIELDS, [
    'xmlRequest' => new CURLStringFile($xml, 'file.xml'),
    #     ^                              ^       ^
    #     |                              |       `-- upload filename
    # name of the form-field             |
    #                                    |
    #                   string contents of the file (plain binary data)
]);

    Cf. CURLStringFile; answer to Send string as a file using curl and php and curl_setopt(CURLOPT_POSTFIELDS) (it already mentions CURLStringFile); CURLStringFile is available since PHP 8.1.

    PHP CURL File Upload from Pathname

    If that is not the case, and you want to upload the file from the file-system (instead of the string data), this is how it works in PHP Curl tailored to your -F [email protected] curl(1) option and argument example:

    curl_setopt($ch, CURLOPT_POSTFIELDS, [
    'xmlRequest' => new CURLFile('file.xml'),
    #     ^                          ^
    #     |                          `-- upload filename
    # name of the form-field  
]);

    Cf. CURLFile; CURLFile is available since PHP 5.5.

    Let's continue with the next lines:

    1. curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml')); - NOT OK, the original curl command line does not have it. Remove it.

    2. curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); - NOT OK, the original curl command line does not have it. Remove it.

    3. curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); - NOT OK, the original curl command line does not have it. Remove it.

    4. curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); - NOT OK, the original curl command line does not have it. Remove it.

    As you have already written, you're not yet interested in converting the redirect to file (the > result.csv at the end of the command-line), so this should already be it for the 500 error you're facing.

    Discussion

    The technique to compare the Curl command-line with the PHP Curl code is a good procedure as it allows a straight forward way for troubleshooting and can always be completed.

    It mustn't mean that it will solve all problems; however, if completed and an error persists, it can be more easily localized and configuration differences between curl on the command-line and the PHP Curl extension identified. For such problems, it is often necessary to review all the curl_setopt() calls (and their order), which is already part of the procedure and therefore enables you to handle higher level problems as well.

    The answer given may also show how to port (translate) a Curl command-line to PHP Curl, however, it does not show how to find the right PHP Curl options (curl_setopt() has the listing of all the options) and their appropriate order (setting some options influence or reset others) in detail. You can also find quite some Q&A on site about the topic in general, both for Curl on the command-line and in PHP Curl.

    For example, PHP - Debugging Curl shows how to obtain verbose and connection information for PHP Curl.

    Also, tools exist to genereate boilerplate PHP Code from a curl command-line, for example Curl-to-PHP, but watch out, such tooling can be limited (this one is from 2017, in PHP that date is the year of the PHP 7.2 release on 30th of November and with that the PHP Curl extension required libcurl version 7.10.5 which is quite dated).

    Extra: If you've got Composer in your tool-belt, you can run composer diagnose which shows the PHP version you've got at hand on the command-line as well as the Curl version of the PHP Curl extension (ext-curl is its name of the platform package in Composer):

    > composer diagnose
...
PHP version: 8.2.8
...
cURL version: 7.81.0 libz 1.2.11 ssl OpenSSL/3.0.2
...

    (This is similar to php_version() and curl_version() which you can make use of with PHP itself.)