Azure Storage Services Api Call: How to use NextPartitionKey and NextRowKey?

Question

The scenario is as follows: The current MVC5 project provides several views the data of which is retrieved by using various API calls. These calls are carried out using only https://[some_url] strings, replacing variables if required.

All calls return JSON strings which are converted into objects based on MVC models. I would also like to point out that the application as such does not feature any databases, since it serves as one step in a chain of processes, passing the objects created to the next application(s).

One call is aimed at an Azure Table Storage. This particular table contains a couple of thousand entries and is growing steadily. Using an "ordinary" API call, such as https://api123.sample.net/application/[...]/ returns only the first 1,000 entries.

This SO post, was my starting point, suggesting to modify the URL by adding the query parameters NextParitionKey and NextRowKey. A comment in the aforementioned SO post also pointed towards this Microsoft Doc, suggesting the same approach of adding parameters to the query.

Since the call I've been using returned x-ms-continuation-NextPartitionKey and x-ms-continuation-NextRowKey in the header, I figured I should use these to modify my query URL. I was able to modify the URL and also got a JSON result upon entering the parameterized query in Postman.

However, and this is the actual question, despite using query parameters, the result of the API call returned only 1,000. Comparing the results of both a parameterized URL call and the standard call used before, revealed that the JSON files were indeed different.

I figure one cannot change this limitation of 1,000 entries per call, but how would I use a URL-based API call to retrieve all entries from this particular database, if possible at all (cf. methods below)? I mean if using the parameters also returns only 1,000 entries per call, how is it possible to "chain" the calls and figure out the last database entry to terminated the entire process.

I assume that I cannot use the URL-based approach in this scenario. Would it be required to extract the actual API call into a proper method (links would be appreciated)?

As a piece of additional information, please consider the following methods (note: slight modifications):

public static string GetJsonDataFromApi()
        {
            var jsonData = "";
            var apiKey = System.Configuration.ConfigurationManager.AppSettings["ApiKey"];
            var url = BuildWebApiUrl();

            if (IsNullOrWhiteSpace(url) && IsNullOrEmpty(url))
                return jsonData;

            var webClient = new WebClient
            {
                Encoding = Encoding.UTF8,
            };

            using (webClient)
            {
                ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
                webClient.Headers.Add("subscription-key", apiKey);
                try
                {
                    jsonData = webClient.DownloadString(url);
                }
                catch (WebException e)
                {

                }
                return !IsNullOrEmpty(jsonData) ? jsonData : "";
            }
        }

private static string BuildWebApiUrl()
        {
            var tableName = System.Configuration.ConfigurationManager.AppSettings["Table"];
            [some more System.ConfigurationVariables]

            return string.Format(urlTemplate, tableName, nextPartitionKey, nextRowKey);
        }

Answer 1

Regarding the issue, I think it is a better way that we write our code to query all entities in Azure table.
If we want to query all entities in Azure table, we can use the SDk Microsoft.Azure.Cosmos.Table to implement it.

var acc = new Microsoft.Azure.Cosmos.Table.CloudStorageAccount(
                         new StorageCredentials("account name", "account key"), true);
            var tableClient = acc.CreateCloudTableClient();
            var table = tableClient.GetTableReference("table name");
            TableContinuationToken token = null;
            var entities = new List<MyEntity>();
            do
            {
                var queryResult = table.ExecuteQuerySegmented(new TableQuery<MyEntity>(), token);
                entities.AddRange(queryResult.Results);
                token = queryResult.ContinuationToken;
            } while (token != null);