ExecuteQuerySegmentedAsync is not working when the Azure table has more records

Question

I'm trying to read the records from Azure table storage. I have a simple query while pulling the records

var isPagination = true;
var  combinedFilter = "groupCode eq '9THS'";
var query = new TableQuery<AzStorageEntityAdapter<T>>().Where(combinedFilter);

TableRequestOptions tableRequestOptions = new TableRequestOptions() 
                                            { ServerTimeout = TimeSpan.FromSeconds(90) };

do
{
    var segments = await table.ExecuteQuerySegmentedAsync(query, continuationToken, tableRequestOptions, null);
    currentPageResult.AddRange(segments.Results.Select(s => s.InnerObject).ToList());
    continuationToken = segments.ContinuationToken;
}  while (continuationToken != null && !isPagination);;

It was working till the azure table had less number of records(10000) with say 3 to 4 distinct Groupcodes.

When the table size increased over 200000 records, the search will not return any records (i.e) segments.Results has zero records, but the continuationToken has values.

If I replace the the ExecuteQuerySegmentedAsync with ExecuteQuery it returns the expected records. I tried to add ServerTimeout, MaximumExecutionTime nothing helped.

Whats wrong here ?

Answer 1

There's nothing wrong here :). It is expected behavior.

From Query Timeout and Pagination:

A query against the Table service may return a maximum of 1,000 items at one time and may execute for a maximum of five seconds. If the result set contains more than 1,000 items, if the query did not complete within five seconds, or if the query crosses the partition boundary, the response includes headers which provide the developer with continuation tokens to use in order to resume the query at the next item in the result set. Continuation token headers may be returned for a Query Tables operation or a Query Entities operation.

Basically your query is doing a full table scan as you have not specified any PartitionKey in your query and it tries to search for matching records by going from top of the table to the bottom. Since it is not finding any matching entities in 5 seconds, it is simply returning the continuation token.

As to why ExecuteQuery is working is because it internally handles the continuation token. You can confirm it by tracing the request/response in Fiddler.