Search code examples
amazon-dynamodb

Updating whole Database in DynamoDB with FilterExpression and ExclusiveStartKey

We have a database in AWS DynamoDB and want to add a field to all items　where it's currently undefined. I was wondering what the best way would be to go about that. I am considering two options:

  1. use scan with filter expression attribute_not_exists(myNewField) and repeat until it returns no results anymore
  2. use scan without filter, check logic later and iterate over all items with ExclusiveStartKey

My question is which one of these would be better or if either could cause problems

The original reason I wanted to ask this question to ask if I could use filter expression attribute_not_exists(myNewField), then add myNewField with an update expression and if I then use ExclusiveStartKey to get the next batch (still with attribute_not_exists(myNewField)) will it give me the correct next batch, will it start over or will there be a some items skipped. Or if someone could share a source to read about this.

Solution

  • You use the LastEvaluatedKey to determine how far you've scanned through the table, it's a pointer to the last item in the page you returned.

    When you set that as the ExclusiveStartKey then you pick up from where the last item read, exclusive of that item.

    What that means is that either option is the same, there is no difference in cost or performance and both will work. However, applying the filter on DynamoDB simplifies your client side code as DynamoDB is responsible for applying the filter logic.