Search code examples
phpshell-execlong-running-processes

PHP and background processes

I am developing a mass-mailing system. At a time we send 2-4K emails, the email contacts are imported using PHPexcel library at same quantity of emails. Last night when we are sending 2k emails we get the "500 internal server" error.

I think I should develop the new process for email handling and the contacts importing, am I right? If so, how should I do it? Is there any other way to overcome such 500 errors?

The PHP script is called by web browser and browser loads it for 5-10 minutes and then 500 error occurs. I am using the PHPMailer library for sending the mail.

Solution

  • Calling a long-running PHP script from a web browser is not really the same as running PHP in the background. That will lock up an Apache thread, and is likely to be subject to whatever timeouts PHP has configured. My guess is the timeout is being hit before the send has completed.

    It would be better to do this on a cron. Here are some general pointers:

    • Every ten minutes, select the next unsent set of email addresses from your data store, maybe 100 of them.
    • Send an email to each one, logging to a database what you have done
    • Pause for a few seconds. This is helpful as it makes it less likely your mail will be directed to spam bins
    • If your script has been running for more than five minutes, exit (it will do the next set of email addresses on the next cron call)
    • Otherwise, loop back to the start

    That will be much more reliable. For bonus points, write a web page to show you what emails are sent and which are still waiting. Also, you may wish to use a third-party mailing system like MailChimp, to increase your delivery reliability. Make sure all of your recipients really have opted into receiving email from you.

    I've suggested the script should batch in groups of 100, run for five minutes, be called every ten minutes, and pause for a few seconds after each send -- but these are just examples. If you don't mind sending more slowly (e.g. overnight) then you can change these figures to suit. Sending more slowly is generally more reliable, so do this if you can.