Search code examples
amazon-dynamodbnosql

Good practices for DynamoDb for data retrieval


I am evaluating DynamoDB for new project as potential db. I would appreciate your input for following use case and solution.

Use Case: User provides multiple device ids to find data about devices. API should return all devices that have device ids that fully matches those entered by User.

Algorithm: In API call send multiple DynamoDB requests to get data. Each request gets one device by device_id that is partition key in table. Once Db returns data, api combines data into one data structure that is returned to UI.

My concern is that I would have to get multiple db requests. Coming from SQL world, same functionality can be achieved by 1 query so that db is accessed only once:

select * from device where device_id in ('123','234', 345')

Questions:

  1. Is DynamoDB data retrieval by partition key is so fast/cheap that multiple data accesses like example above is justified?
  2. Is there another approach/algorithm that I should use to get multiple items by their ids in one query?

Solution. Answer below with Execute statement suggestion is good option. However, I am trying to stick with existing pattern to retrieve data using DDB resource and query data with it.

I ended up updating data model so that there is only 1 query to DB to get data. I added attribute search_id that duplicates device_id value from sort key. This enabled to filter data with logical operator IN . (Right now this operator is not visible in foptions for filter in DynamoDb UI which was the reason I did not thought of it in the first place). Query string ended up looking like this:

 aws_query = {
            'Limit': self.take,
            'Select': 'ALL_ATTRIBUTES',
            'ScanIndexForward': False,  
            'TableName': 'Devices',
            'KeyConditionExpression': Key('type').eq('device'),
            'FilterExpression': Attr('search_id').is_in(device_ids)
        }

Solution

  • You have several ways to get the data back in a single request:

    1. If your table is partition key only, you can use BatchGetItem to obtain up to 100 items in a single request
    2. If you have a sort key, but you still want all of the data for each device, you can use ExecuteStatement where you can use the exactly SQL syntax you shared and get data for up to 50 devices in a single request.