Why is my dynamoDB queries taking so much time?

Question

I have a lambda that reaches to a dynamoDB, with a very simple CRUD operations. each of the operations is taking between 200-500~ ms, which is very odd since I have a very small amount of data (3 tables with around 50 entries for each).

I used the following code to measure the execution time of dynamoDB queries:

const queryStartTime = Date.now();
const data = await query();
console.log(`query time: ${Date.now() - queryStartTime}`);

query is a parameter in some abstract method I created to wrap all queries. for example, a query will look something this:

async () => {
            const data = await this.database.send(new GetCommand(params));

            if (!data.Item)
                throw new ResourceNotFoundException({
                    message: 'the returned item is undefined',
                    $metadata: data.$metadata
                });

            return { record: TaskRecordSchema.parse(data.Item) };
        }

I tried first to check if using scan is the source of the problem, but I compared the amount of time it takes for a ScanCommand and GetCommand and it turns out to be kind of the same.

In addition, increasing the memory for the lambda also did not work (went from 128mb to 512mb).

Answer 1

The issue here is with the rate of requests. As you are measuring full round trip latency you will have the following overheads:

1. TCP Connection + Establishment
2. Client Initialization
3. DynamoDB metadata cache misses (extra hops required to obtain AuthZ and AuthN)
4. DynamoDB metadata cache misses for the locations of your partitions and data (extra hops required)

DynamoDB offers single digit millisecond response times, which is server side latency and not full round trips.

200 - 500 ms latency looks to be directly aligned with infrequent access.

1. Increase your request rate
2. Be sure to initialize your clients outside of the request handler.
3. Also be sure to set TCP Keep Alive to True.