Server-side paging with WCF OData and GetContinuation() method

Question

I am using the QueryOperationResponse GetContinuation() method to attempt and page through a very large data set read off an odata wcf feed. I am using some OData helper libraries acquired from microsoft.

The problem is that the paging process seems to get stuck in an infinite loop, and never finishes. For example if I set the page size to 10000 records and there are 80000 records to retrieve, I observe that the loop continues for well over 8 iterations, when it should be finished.

Below is the class that queries the service and implements the paging (at the bottom). I also observed that the 'OriginalString' of the NextLinkUri never changes with each iteration, which I think is wrong? Hopefully I'm just missing something really obvious, I think this is the correct way to page:

    private static IList<dynamic> Get(string serviceUri, NameValueCollection queryOptions, IAuthenticationScheme authenticationScheme)
    {
        string baseUri;
        string entitySet;
        string entityKey;
        string queryString;
        ValidateServiceUri(serviceUri, out baseUri, out entitySet, out entityKey, out queryString);
        string resource = !string.IsNullOrEmpty(entityKey) ? entitySet + "(" + entityKey + ")" : entitySet;

        DataServiceContext context = new DataServiceContext(new Uri(baseUri));
        context.IgnoreMissingProperties = true;

        DataServiceContextHandler handler = new DataServiceContextHandler(authenticationScheme);
        handler.HandleGet(context);

        DataServiceQuery<EntryProxyObject> query = context.CreateQuery<EntryProxyObject>(resource);

        NameValueCollection options = HttpUtility.ParseQueryString(queryString);
        options.Add(queryOptions);

        foreach (string key in options.AllKeys)
        {
            query = query.AddQueryOption(key, options[key]);
        }

        QueryOperationResponse<EntryProxyObject> response = query.Execute() as QueryOperationResponse<EntryProxyObject>;

        IList<dynamic> result;
        if (options["$inlinecount"] == "allpages")
        {
            result = new DynamicEntityCollection(response.TotalCount) as IList<dynamic>;
        }
        else
        {
            result = new List<dynamic>();
        }

        foreach (EntryProxyObject proxy in response)
        {
            DynamicEntity entity = new DynamicEntity(proxy.Properties);
            result.Add(entity);
        }

        while (response.GetContinuation() != null)
        {
            DataServiceQueryContinuation<EntryProxyObject> continuation = response.GetContinuation();
            QueryOperationResponse<EntryProxyObject> nextResponse = context.Execute<EntryProxyObject>(continuation);

            Console.WriteLine("Uri: " + continuation.NextLinkUri.OriginalString); 

            foreach (EntryProxyObject proxy in nextResponse)
            {
                DynamicEntity entity = new DynamicEntity(proxy.Properties);
                result.Add(entity);
            }
        }

        return result;
    }

This is how I call the method:

return OData.Get("http://xxx.x.xxx.xx/MyDataService.svc/MyProducts", "$filter=Index gt 0");

Answer 1

You are always looking at the continuation of the first response. That stays constant (for obvious reasons). You need to look at the continuation of the nextResponse in your code to figure out if there's more data to read and to get the continuation to get the next page. The code above reads the first page and then reads the second page over and over again.