I have a website in shared hosting with the Entry Processes limit of 30. I fetch data periodically from another URL using cURL function on a PHP cron job. The relevant code is as below.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "GET");
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
Most of the time my website runs pretty well with only around 4-5 entry processes being used. Whenever this target $url is doesn't response for some reason (and it happens frequently). I quickly get into Entry Process limit and all further requests are denied.
The CURLOPT_CONNECTTIMEOUT doesn't seem to be working as expected. How can I avoid this situation ? I have checked other cURL options but none seem to be working.
A 10 seconds timeout is very long. Depending on how many requests you serve and if all your requests trigger this call, all your available processes could be just waiting for answers from that server.
You could consider lowering that number.
There is also a second timeout for curl CURLOPT_TIMEOUT. Try setting this in addition. If the connection to the server is done within 10 seconds and then the server takes 60 seconds to serve your request, the current timeout doesn't help you because it only limits the connect time.
If you don't want to be dependent on what cURL is doing you could also set a time limit for the php process itself with set_time_limit(). If you set that to e. g. 30 seconds, php will stop the execution after that time no matter if cURL is done yet or not. This should be done before the curl calls.