I have an AWS Lambda function, which subscribes to a DynamoDB stream and is configured with an SQS dead letter queue (DLQ). I can see that the correct queue is configured in the Management Console. Also I took care to give my function permissions for sqs:SendMessage on my DLQ.
The subscription works, but still "hangs" on invocation errors as if no DLQ were configured. I.e., if there is a message, which leads to an unhandled exception, the function continues to retry this message until it's dropped from the stream. I can see that the number of invocation errors rises, and no DLQ errors are shown in the function's Cloudwatch dashboard. The SQS queue remains empty.
What I want is that failed messages get forwarded to my DLQ and the subscription continues to the next message. Any ideas?
As Jonathan Seed said below, DLQ's currently don't work with stream-based subscriptions. AWS Support confirmed that they're working on implementing this though.
I believe this is because DynamoDB streams are stream based event sources. The lambda documentation states that when dealing with stream based event sources "if a Lambda function fails, AWS Lambda attempts to process the erring batch of records until the time the data expires"
From my understanding, the lambda function will retry until the event is either processed successfully or expires and disappears from the stream, the event is never "discarded" by the lambda function, as they are in non-stream based event sources.
You may have to implement your own failure handling as a part of your main lambda function if you wish to discard certain events, posting the event manually to a queue/topic and returning succesfully.