aws sns invoking lambda multiple times

Question

My AWS application does not allows duplicates. In my application (fan-out) SNS triggers multiple lambda services. Since SNS follows at least once execution, there are chances for triggering same service multiple times.

1. If i use SNS FIFO fixes duplicate issue? or any best alternatives?
2. With SNS FIFO we can subscribe only SQS, any alternatives to trigger lambda directly?

My intention is to SNS==> different lambdas (based on input msg type, without duplicates)

Thanks in advance, Anil

Answer 1

1. Yes, if you provide a deduplication ID or if you enable content-based message deduplication on the topic. Also, you will have to have an SQS FIFO queue.

The AWS docs has this to say about the deduplication:

Amazon SNS FIFO topics and Amazon SQS FIFO queues support message deduplication, which provides exactly-once message delivery and processing as long as the following conditions are met:

• The subscribed SQS FIFO queue exists and has permissions that allow the Amazon SNS service principal to deliver messages to the queue.
• The SQS FIFO queue consumer processes the message and deletes it from the queue before the visibility timeout expires.
• The Amazon SNS subscription topic has no message filtering. When you configure message filtering, SNS FIFO topics support at-most-once delivery, as messages can be filtered out based on your subscription filter policies.
• There are no network disruptions that prevent acknowledgment of the message delivery.

2. The answer should be obvious, for this one. No, at this this point in time, you can have only SQS FIFO as a subscriber for the topic. The AWS documentation is pretty specific on this:

To fan out messages from Amazon SNS FIFO topics to AWS Lambda functions, extra steps are required. First, subscribe Amazon SQS FIFO queues to the topic. Then configure the queues to trigger the functions.