Multi-threading to speed up an email-sending application

Question

I have built an application to send email mailers for a website through Amazon SES. It is coded in C#.

Each email takes .3 seconds to send via the Amazon SES API. That means, using a single-threaded application, I can only send 3 emails per second.

I have implemented a producer/consumer, multi-threaded application with 1 producer to query customize the emails for each customer, and 25 consumers to pull from the queue and send the emails.

My multi-threaded application sends 12 emails per second (a quadruple speed increase). I would have expected a greater speed increase from a 25-thread application.

My question is: How much can I really speed up the sending of a mailer on a single-processor machine? Do my gains seem reasonable, or is my speed problem more likely due to coding than to the computer's inability to process the emails mroe quickly?

Thanks in advance!

UPDATE: In case others are facing the same issue.... connecting to AWS in order to send the email takes up a lot of time. The following thread on AWS Developer forums gives some insight (You may need to scroll down to get to the more useful posts).

https://forums.aws.amazon.com/thread.jspa?threadID=78737

Answer 1

My question is: How much can I really speed up the sending of a mailer on a single-processor machine? Do my gains seem reasonable, or is my speed problem more likely due to coding than to the computer's inability to process the emails more quickly?

Broadly speaking, a 4x speedup for a 25x increase in thread counts isn't outrageous, but it's not great, either.

A single CPU will only become a bottleneck when your CPU usage is high. You can tell whether that's an issue for you by looking at total CPU use when your app is running. In theory, sending bulk emails should be an I/O limited operation; if that's not the case for you, then your code may have issues.

Although I haven't used Amazon SES, I know that other Amazon products definitely use various forms of bandwidth / request throttling. It's possible (likely) that your throughput is being limited more by Amazon than by your app.

I wrote a high-performance bulk mail app a while back, and what I did was:

1. Used async I/O as much as possible, in addition to multiple threads. That way, if one request is slow, it doesn't consume an entire thread.
2. Sent the email directly to the end servers, rather than through an intermediate gateway. That required using P/Invoke to call DNS to retrieve the requisite MX or A records. After that, I used the standard SmtpClient class (which has a SendAsync method) to actually send the mail.

This approach also lets me see and record errors when sending the mail, which in turn provides better feedback to the users. The alternative is to rely on receiving and parsing error mail from the gateway server, which is error-prone, to say the least.