Emailing entire userbase in one PHP script?

Question

I'm wondering how do people handle emailing their userbase. I'm using PHP and SendGrid is handling the emails. The use case is a newsletter.

When I send myself 50 emails in testing the script already takes quite a long time and I worry that the application will time out if go for more emails at a time.

I don't really want to send myself 100, then 1,000, then 10,000 emails to see what the upper limit is testing this out.

I'm running this on a medium EC2 Linux if that is helpful.

Answer 1

There are several ways to accomplish this with SendGrid.

Send Emails in Parallel

It's likely that you're currently sending your emails in a series (i.e. send one email, wait for it to send, then send the next). This means if it takes you 1 second to send an email, it will take you 100 seconds to send one hundred.

Luckily, PHP's cURL Library allows you to send many POST requests in parallel, rather than in a series.

// An array of the emails you want to send to:
$emails = array("[email protected]", "[email protected]");

$credentials = array("api_user" => YOUR_SENDGRID_USERNAME, "api_key" => YOUR_SENDGRID_PASSWORD);

$multi_handle = curl_multi_init();
$handles = array();
// Loop through each email and create a cURL Handle for the email
// The cURL handle will POST an email to SendGrid with the proper info
foreach ($emails as $key => $email) {
    // Create an email data object with your credentials in it,
    // we'll POST this to SendGrid
    $email_data = $credentials;

    // Fill out all the information you want for the email
    $email_data = array(
        "to" => $email,
        "from" => "[email protected]",
        "subject" => generate_subject(),
        "html" => generate_html(),
        "text" => generate_text()
    );

    $handles[$key] = curl_init();
    curl_setopt($handles[$key], CURLOPT_URL, "https://api.sendgrid.com/api/mail.send.json");
    curl_setopt($handles[$key], CURLOPT_POST, 1);
    curl_setopt($handles[$key], CURLOPT_POSTFIELDS, $email_data);

    curl_setopt($handles[$key], CURLOPT_RETURNTRANSFER, true);
    curl_multi_add_handle($multi_handle, $handles[$key]);
}

$running = null;
// Run through each cURL handle and execute it.
do {
    curl_multi_exec($multi_handle, $running);
} while ($running);

curl_multi_close($multi_handle);

This method works well if you have lots of hyper-unique emails that you want to send, however it still results in a bunch of POST requests from your server, which are slow and resource consuming. There's often a better way.

Send Emails Using the SMTPAPI

SendGrid has an SMTPAPI which allows you to send up to 10,000 emails with one POST request using the to parameter. You can make these emails semi-unique by using substitution tags.

For this you'd do something like the following:

// An array of the emails you want to send to:
$emails = array("[email protected]", "[email protected]");

// Encode it into the SendGrid SMTPAPI Format
$smtpapi = array( "to" =>  $emails );

$params = array(
    "api_user"  => YOUR_SENDGRID_USERNAME,
    "api_key"   => YOUR_SENDGRID_PASSWORD,
    "to"        => "[email protected]",
    "subject"   => "testing from curl",
    "html"      => "testing body",
    "text"      => "testing body",
    "from"      => "[email protected]",
    "x-smtpapi" => json_encode($smtpapi);
  );


$request =  "https://api.sendgrid.com/api/mail.send.json";

// Generate curl request
$session = curl_init($request);
// Tell curl to use HTTP POST
curl_setopt ($session, CURLOPT_POST, true);
// Tell curl that this is the body of the POST
curl_setopt ($session, CURLOPT_POSTFIELDS, $params);
// Tell curl not to return headers, but do return the response
curl_setopt($session, CURLOPT_HEADER, false);
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);

// obtain response
$response = curl_exec($session);
curl_close($session);

However, it's probably better to do this with SendGrid's PHP Library

$emails = array("[email protected]", "[email protected]");

$sendgrid = new SendGrid(YOUR_SENDGRID_USERNAME, YOUR_SENDGRID_PASSWORD);
$email    = new SendGrid\Email();
$email->setTos($emails)->
       setFrom('[email protected]')->
       setSubject('Subject goes here')->
       setText('Hello World!')->
       setHtml('<strong>Hello World!</strong>');

$sendgrid->send($email);

Queueing

Finally, you could utilize a queue and send all the emails from a different process to speed up your execution time, information on how to do that is in this StackOverflow response.