How to prevent SendGrid throttling when sending multiple emails?

Question

For a client of mine I am currently building a 'payslip' export. The export has to be emailed to around 90 of their employees. Each email contains a unique attachment with their own payslip in it.

My current solution is to use the SendGrid.Helpers.Mail.MailHelper package to create an email using the CreateSingleTemplateEmail method. I then use the AddAttachment method to include an attachment. Finally, I send the email by using the SendGridClient.SendEmailAsync method.

All of the above logic is in a foreach loop that iterates over all 90 employees. The problem however, is that the first 15 emails are sent instantaneously, after which the API seems to get throttled. The application is built in a simple Azure Static Web App (cheaper). So there's no real possibility for complex solutions, except if we start paying for more complex solutions. Our goal is to keep this on the cheap side.

After around two minutes, the Azure Function times out and it stops sending the emails. I've been searching the internet for possible solutions, but I haven't really found a good solution that includes attachments.

Do you guys have any suggestions for me, or is this not possible at all?

Thanks! Thomas

Answer 1

What if you did not send all of the mails from within one function, but make use of the power of Azure Functions and use Queues, basically to implement Queued Load Levelling. Since I do not know your code, I won't come up with an actual example, but basically, you could do the following

Each of the lanes is a function.

• The first one is responsible only for retrieving the payslip data (from a database I guess). It iterates over all the employees and writes the data for each employee to a queue.
• The second function is triggered by this very queue and generates a PDF from the payslip data for one employee at a time. This PDF along with some metadata is then written to a second queue (or to a blob container, you maybe will have to experiment which one is the most useful for you, they both have their advantages I guess).
• Then a third function would be triggered by the second queue (or by the blob storage), generates an email for that one PDF (based on the metadata) and sends it via SendGrid.

This way even if the API throttles (which seems to be unlikely, based on the comments on the question), each executed function would still finish in a time that is not prone to experience a timeout. This also seems to be a more Cloud (or Serverless) native approach to me than doing it all in one function.