Search code examples
phpcurlcurl-multi

PHP curl_multi_getcontent returns null


I have been following this tutorial on how to use curl_multi. http://arguments.callee.info/2010/02/21/multiple-curl-requests-with-php/

I can't tell what I am doing wrong, but curl_multi_getcontent is returning null. It is suppose to return JSON. I know it is not the mysql call as I had it working with a while loop and standard curl_exec, but The page was taking too long to load. (I've changed some of the setopt details for security)

Relevant PHP Code snippet. I do close the while loop in the end.

$i = 0;
$ch = array();
$mh = curl_multi_init();
while($row = $result->fetch_object()){
   $ch[$i] = curl_init();
   curl_setopt($ch[$i], CURLOPT_CAINFO, 'cacert.pem');
   curl_setopt($ch[$i], CURLOPT_USERPWD, "$username:$password");
   curl_setopt($ch[$i], CURLOPT_RETURNTRANSFER, true); 
   curl_setopt($ch[$i], CURLOPT_URL, 'https://mysite.com/search/'.$row->username.'/');
   curl_multi_add_handle($mh, $ch[$i]);
   $i++;
}
$running = 0;
do {
    curl_multi_exec($mh, $running);
} while ($running > 0);
$result->data_seek(0);
$i = 0;
while ($row = $result->fetch_object()) {
    $data = curl_multi_getcontent($ch[$i]);
    $json_data = json_decode($data);
    var_dump($json_data);

EDIT

Here is the code that currently works, but causes the page to load too slowly

$ch = curl_init();
curl_setopt($ch, CURLOPT_CAINFO, 'cacert.pem');
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); 
while($row = $result->fetch_object()){
   curl_setopt($ch, CURLOPT_URL, 'https://mysite.com/search/'.$row->username.'/');
   $data = curl_exec($ch);
   $json_data = json_decode($data);
   var_dump($json_data);
}

Solution

  • I'm wondering:

    $i = 0;
    while ($row = $result->fetch_object()) {
        $data = curl_multi_getcontent($ch[$i]);
        $json_data = json_decode($data);
        var_dump($json_data);
    

    Are you forgetting to increment $i? If so, you already grabbed the content for $ch[0], and then you call curl_multi_getcontent again.

    Also, I've written a blog post covering concurrent requests with PHP's cURL extension, and it contains a general function for curl multi requests. You could call this function in the following manner:

    $responses = multi([
        $requests = [
            ['url' => 'https://example.com/search/username1/'],
            ['url' => 'https://example.com/search/username2/'],
            ['url' => 'https://example.com/search/username3/']
        ]
        $opts = [
            CURLOPT_CAINFO => 'cacert.pem',
            CURLOPT_USERPWD => "username:password"
        ]
    ]);
    

    Then, you cycle through the responses array:

    foreach ($responses as $response) {
        if ($response['error']) {
            // handle error
            continue;
        }
        // check for empty response
        if ($response['data'] === null) {
            // examine $response['info']
            continue;
        }
        // handle data
        $data = json_decode($response['data']);
        // do something
    }
    

    Using this function, you could make a simple test of accessing https sites using the following call:

    multi(
        $requests = [
            'google' => ['url' => 'https://www.google.com'],
            'linkedin' => ['url'=> 'https://www.linkedin.com/']
        ],
        $opts = [
            CURLOPT_CAINFO => '/path/to/your/cacert.pem',
            CURLOPT_SSL_VERIFYPEER => true
        ]
    );