Preserve FIFO processing with an AWS SQS FIFO Queue as a lambda trigger event source and Step function processing

Question

Is it possible through configuration to setup a SQS FIFO queue with a lambda trigger attached to use the lambda to call a step function while preserving order all the way through processing? So this would mean not firing off multiple step functions in parallel in the running state, but preserving the FIFO processing so any new step function executions would need to wait until the step function completes before processing the next batch of records.

Is this something I can achieve through configuration or is another route necessary ?

Answer 1

I don't think this would be possible if you're using the builtin integration between SQS and Lambda - as soon as your function execution exits (which will be as soon as it is finished starting the step function), AWS Lambda will fetch the next message.

If your execution time is relatively short, and you are willing to pay the Lambda execution time, you could poll or wait on the Step Function to complete in your Lambda function before returning. This could be expensive however if you have a lot of messages to process or your Step function takes some time to complete.