Search code examples
phpcurlparallel-processingcurl-multi

PHP: Parallel cURL Performance - RollingCurl vs ParallelCurl

I have a large amount of cURL requests to process. There are a couple libraries I've found to help process them parallel, the main ones are RollingCurl ( http://code.google.com/p/rolling-curl/ ) and Parallel Curl ( https://github.com/petewarden/ParallelCurl ).

Does anyone have experience with these libraries, and if so do you know which one performs better?

Thanks!

UPDATE: For anyone who may stumble upon this later, if you're using Rolling Curl's group functionality it seems to break when handling a large number of requests (50k+)

Solution

  • Performance is not the issue since they are both wrapping standard but functionality

    • RollingCurl support group process which you can also achieve with simple array

    • Parallel Curl i slight ans straight froward .. with optional custom curl options

    • They Both Support callback

    Both are nice it depends on your need .....