Search code examples
amazon-dynamodb-dax

How dynamoDB + DAX work with timeseries?


I wonder how DAX works with time-series. I want to insert some data every minute, add TTL to remove it after 14 days and get last 3 hours of data after each insert:

  • insert 1KB each minute
  • expire after 14 days
  • after each insert read data for the last 3 hours

3 hours is 180 minutes, so most of the time I need the last 180 items. Sometimes data is not coming for some time, so there may be less than 180 items.

So there are 20,160 items ±19MB of data for 14 days. How much DAX I will use while fetching the last 3 hours of data every minute? Will it be 19MB or 180KB?

let params = {
    TableName: 'prod_server_data',
    KeyConditionExpression: 's = :server_id and t between :time_from and :time_to',
    ExpressionAttributeValues: {
      ':server_id': serverId, // string
      ':time_from': from,     // timestamp
      ':time_to': to,         // timestamp
    },
    ReturnConsumedCapacity: 'TOTAL',
    ScanIndexForward: false,
    Limit: 1440, // 24h*60 = 1440. 1 check every 1 min
  };

  const queryResult = await dynamo.query(params).promise();

Solution

  • DAX caches items and queries separately, and the query cache stores the entire response, keyed by the parameters. In this case, set the query TTL to 1 minute, and make sure that :time_from and :time_to only have 1 minute resolution.

    If you only call query once per minute, than you won't see much benefit from DAX (since it will have to go to DynamoDB every time to refresh).

    If you call query multiple times per minute but only expect the data to update every minute (i.e. repeatedly refreshing a dashboard) there will only be 1 call to DynamoDB every minute to refresh and all other requests will be served from the cache.