Return more than 1000 entities of a Windows Azure Table query

Question

My question is exactly this one. However, the Azure Storage API has changed and all answers I can find for this question deal with the old version. How do handle queries that return more than 1000 items in the current API version? The query fetching less than 1000 items looks like this:

var query = new TableQuery<TermScoreEntity>()
            .Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal,    Name));

var table = _tableClient.GetTableReference("scores");

foreach (var termScoreEntity in table.ExecuteQuery(query))            
    result.Add(termScoreEntity.RowKey, termScoreEntity.Score);

Answer 1

Have you tried to retrieve more than 1000 entities with your above code? I think use simple ExecuteQuery is enough.

The documentation only said it has 1,000 limitation for pure REST requests, the ExecuteQuery API internally use this rest api, but automatically cut to multiple requests, each of them has a limitation of 1,000.

So if you use the .NET package, you don't need to worry about this limitation, the library already handle this for you. You can refer to this thread for more information.

Here is some key code for ExecuteQuery from Azure SDK, you would find actually the max number is 9223372036854775807L if you don't specify the take number:

internal IEnumerable<DynamicTableEntity> Execute(CloudTableClient client, CloudTable table, TableRequestOptions requestOptions, OperationContext operationContext)
{
  CommonUtility.AssertNotNullOrEmpty("tableName", table.Name);
  TableRequestOptions modifiedOptions = TableRequestOptions.ApplyDefaults(requestOptions, client);
  operationContext = operationContext ?? new OperationContext();
  return CommonUtility.LazyEnumerable<DynamicTableEntity>((Func<IContinuationToken, ResultSegment<DynamicTableEntity>>) (continuationToken =>
  {
    TableQuerySegment<DynamicTableEntity> local_0 = this.ExecuteQuerySegmented((TableContinuationToken) continuationToken, client, table, modifiedOptions, operationContext);
    return new ResultSegment<DynamicTableEntity>(local_0.Results)
    {
      ContinuationToken = (IContinuationToken) local_0.ContinuationToken
    };
  }), this.takeCount.HasValue ? (long) this.takeCount.Value : long.MaxValue);
}