Search code examples
amazon-web-servicesaws-lambdaamazon-dynamodbamazon-dynamodb-streams

DynamoDB stream's tumbling window - Will there be only a single Lambda function used for this?


For the tumbling window concept:

This new feature introduces the concept of a tumbling window, which is a fixed-size, non-overlapping time interval of up to 15 minutes. To use this, you specify a tumbling window duration in the event-source mapping between the stream and the Lambda function. When you apply a tumbling window to a stream, items in the stream are grouped by window and sent to the processing Lambda function. The function returns a state value that is passed to the next invocation of the tumbling window.

After reading this, I understand that we can aggregate dynamo db stream updates for some time, which then would be consumed by a lambda in bulk (also see: Analytics with Tumbling Windows :: My AWS Workshop).

Now if I were to not use this concept, then for each row update, a different instance of lambda would have been created, if the updates were done close enough.

But in this case, I am not sure whether there would be any concept of multiple instances of lambdas working on dynamo db stream updates.

Wanted to confirm whether that would be the case? We need to design our service accordingly.


Solution

  • There will always be multiple instances of Lambda when reading from a DynamoDB Stream. This is due to the relationship between DynamoDB partitions and stream shards and the Lambda conatainers:

    1 : 1 : 1
    DynamoDB Partition : Stream Shard : Lambda Container
    

    So for every partition in your DynamoDB table you will have an active shard and consequently a Lambda invocation.