Search code examples
amazon-dynamodbamazon-kinesisamazon-dynamodb-streamsamazon-kcl

Does Kinesis Client Library consumer support AT_TIMESTAMP as a starting position when using Dynamodb streams kinesis adapter

According to doc Dynamodb streams and Kinesis data streams low-level APIs are similar, but they are not 100 percent identical.

I notice that Dynamodb streams' GetShardIterator is a bit different i.e it doesn’t support AT_TIMESTAMP as a shard iterator type.

So, I presume that the adapter implementation will limit KCL consumer functionalities, and doesn’t allow AT_TIMESTAMP starting position.

Is my reasoning correct? I don’t yet implement it. I prefer to look for another solution if this seems to be a blocking point.

Solution

  • Kinesis Client Library consumer doesn't support AT_TIMESTAMP as a starting position when using Dynamodb streams kinesis adapter:

    https://github.com/awslabs/dynamodb-streams-kinesis-adapter/blob/50750146b1ccc1d87793522780ca21b4e376b2e1/src/main/java/com/amazonaws/services/dynamodbv2/streamsadapter/model/GetShardIteratorRequestAdapter.java#L142