Configure AWS SQS to take runtime exceptions into Dead Letter Queue

Question

I am trying to integrate an Async Design with API Gateway -> SQS -> Lambda.

The gateway receives a request and via the SQS integration at gateway we are passing the messages to SQS. The SQS then passes the messages to Lambda and eventually Lambda does the exectution.

I want to handle runtime errors(validation errors, business errors or runtime exceptions).

Please suggest a method to handle such exception with a dead letter queue.

How can I use SQS as a DLQ to receive messages from Lambda based on HTTP status codes from service (in case of runtime failures)?

I tried configuring a SQS as a DLQ Inside Lambda Async Invocation Configuration but even after getting runtime exception the messages are not going to the SQS-DLQ.

Answer 1

The moment the incoming message lands in SQS, the API gateway should send back the success response (HTTP 202) back to the requester. The lambda function will pick the message from the queue and start processing. This will be a synchronous invocation. You need to attach a DLQ to the SQS and configure the maxReceiveCount as per your requirement. The standard practice is to set the maxReceiveCount to 5. That means in case of failure or exception, the lambda function will re-try 5 times before moving the message in DLQ. However in the handler method of lambda code, the exception need to be thrown back to the caller, otherwise the message will be considered as delivered and will be deleted from SQS. The message will be moved to DLQ automatically after 5th retry without any coding in Lambda.

You may refer to the links below to understand how to configure DLQ.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SetQueueAttributes.html

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html

This is the visual representation of the solution.