Search code examples
amazon-dynamodbdynamodb-queriesaws-sdk-java-2.0

DynamoDB V2 Enhanced SDK to get all records matching a partition key


We have a DynamoDB table with both Partition and Sort keys. In most cases there are many records for each Partition key and we need to get all of them in one query. We are using the V2 Enhanced Java SDK and have followed the examples documented here and here.
We have been looking at the batchGetItem() method and it would make sense to do the following where we specify the Partition Key and ignore the Sort key, since we want to return the records for ALL sort key values, but I get an error: The provided key element does not match the schema.

import software.amazon.awssdk.services.dynamodb.model.*;

List<Integer> keys = List.of(0,1,2,3,4);
ReadBatch.Builder<MyRecord> readBatchBuilder = ReadBatch.builder(MyRecord.class);
keys.forEach(id -> readBatchBuilder.addGetItem(Key.builder().partitionValue(id).build()));
readBatchBuilder.mappedTableResource(table);
ReadBatch readBatch = readBatchBuilder.build();
BatchGetResultPageIterable iterable = enhancedClient.batchGetItem(r -> r.addReadBatch(readBatch));

If I add the sort key specifier, like this, the code works and returns the single record, but I don't know the sort key values in advance, so it does my little good.

List<Integer> keys = List.of(0,1,2,3,4);
ReadBatch.Builder<MyRecord> readBatchBuilder = ReadBatch.builder(MyRecord.class);
keys.forEach(id -> readBatchBuilder.addGetItem(Key.builder().partitionValue(id).sortValue("VALUE").build()));
readBatchBuilder.mappedTableResource(table);
ReadBatch readBatch = readBatchBuilder.build();
BatchGetResultPageIterable iterable = enhancedClient.batchGetItem(r -> r.addReadBatch(readBatch));

Is there a way to use batchGetItem() to query all records for a partition key?


Solution

  • Just spoke with AWS support and they explained that the batchGetItem() will only return singleton records and cannot return groups or lists of records. It was recommended that I use the following, which includes auto-pagination (that is, it handles repeated requests when the 1MB load-limit is reached).

    QueryConditional queryConditional = QueryConditional
            .keyEqualTo(Key.builder().partitionValue(keyID)
                    .build());
    
    List<MyRecord> list = new ArrayList<>();
    for (MyRecord record : table.query(r -> r.queryConditional(queryConditional)).items()) {
            System.out.println("The record id is " + record);
            list.add(record);
    }
    

    Edit 1: I like this syntax better:

    QueryConditional queryConditional = QueryConditional
            .keyEqualTo(Key.builder().partitionValue(provid)
                    .build());
    
    List<Ai2_CohortConfigRecord> list = table.query(r -> r.queryConditional(queryConditional))
            .items()
            .stream()
            .peek(record -> System.out.println("The record id is " + record))
            .collect(Collectors.toList());