.Net Core - DynamoDB ScanAsync method doesn't return data for specific records

Question

I'm having an annoying issue with AWS DynamoDB while trying to access data in a .Net core 6 API. Here is the part of the code that retrieve the data, by user_id:

private readonly AmazonDynamoDBClient _amazonDynamoDB; // assigned by dependency injection through constructor) 
string tableName = _serviceConfiguration.AWS.DynamoDB.TableName;
var result = await _amazonDynamoDB.ScanAsync(new ScanRequest
{
    TableName = tableName,
    ExpressionAttributeValues = new Dictionary<string, AttributeValue>
    {
        {":user_id", new AttributeValue {S = userId_input}},
    },
    FilterExpression = "user_id = :user_id"
});

The strange thing is, for some specific userId_input, I get 0 data (result.Count is 0), while another API can add new item into the DynamoDB database, or I can scan those records from AWS interface. It only happens for some user_id values.

I guess this may not contain enough information, but since it's a commercial project, I need to cover as much as possible. However, if anything is really needed for the troubleshooting process, I will try to provide it. For the issue, do you have any idea of which might be the cause?

Answer 1

The results are paged. You need to keep pulling pages. See https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LowLevelDotNetScanning.html and ExclusiveStartKey specifically.