How to performance tune my table in Dynamo db having DynamoDBAutoGeneratedKey as Hash Key as the PutRequest is getting slow with each insert

Question

I am using dynamo db tables for saving the transactional data for my APIs requests. I am maintaining two tables 1. schedule - with SId as hashkey 2. summary - with DynamoDBAutoGeneratedKey (UUID) as hashkey and SId as an Attribute to it.

schedule table populates a single row per request, whereas the summary table populates 10 items per SId and unique UUID

We are running a load test on these two tables and it is observed that schedule table is performing well but the summary table is consuming a lot of time in PutRequests for the 10 items per call.

Can any one suggest on performance tuning for my summary dynamodb table? Can keeping a UUID as hashkey, slow down the PutItemRequest?

Any help pointers are much appreciated.

Also, we have activated the streams on these tables which is consumed by lambda for cross replication.

Answer 1

Few thing that comes to mind:

• Are you using scans by any chance? This would explain performance degradation, since scans do not exploit any knowledge about how data is organised in DynamoDB and are simply a brute force search. You should avoid using scans since they are inherently slow and expensive.

• Do you have a "hot partition"? You wrote:

1. schedule - with SId as hashkey 2. summary - with DynamoDBAutoGeneratedKey (UUID) as hashkey and SId as an Attribute to it.

Is access to these values uniformly distributed? Do you have items that are accessed more often then others? If so, this may be an issue, if majority of your reads/writes comes to a small subset of ids, than it means that you are flooding a single partition (physical machine) with requests. I would suggest to investigate this as well.

One solution can be to use cache and store frequently accessed items there. You can use either ElasticCache or DAX - a new caching solution in Dynamo.

You can find out more about hot partitions here and here.

• Are you using transactions? You wrote:

I am using dynamo db tables for saving the transactional data

If by this you mean that you are using DynamoDB transactions, you need to read how DynamoDB implements transactions.

Long story short, DynamoDB is storing copies of all items that you update/delete/add when you perform a transaction. Additionally, DynamoDB transactions are expensive and they require 7N+4 writes per transaction, where N is a number of items involved in a transaction.