Dynamo DB - batch-get-item pagination

Question

I'm trying to run a batch-get-item from CLI and getting an error that I cannot pass more than 100 keys:

failed to satisfy constraint: Member must have length less than or equal to 100

This is the command I use

aws dynamodb batch-get-item \
--request-items file://request-items.json \
--return-consumed-capacity TOTAL > user_table_output.txt

I would like to know if there's a way I can add pagination to my query? or is there another way I can run the query? I have ~4000 keys which I need to query.

Thanks in advance.

Answer 1

You'll have to break your keys down into batches of no more than 100. Also, keep in mind that the response may not include all the items you requested if the size of the items being returned exceeds 16MB. If that happens the response will include UnprocessedKeys, which can be used to request the keys that were not retrieved.

BatchGetItem has information about the API, or you can view the AWS CLI v2 doc here.