Fetching and paginating all data by AtTask RESTful API

Question

I am trying to make a Get request to AtTask RESTful API to grab 1000s of project objects. ATask API documentation gives some hint on how to do this. I need to paginate the number of objects that I am going to request. Imagine that I need to grab whatever object that is available for the past 7 days, how can I change the following code to achieve this since I don't know the exact number of objects, so I don't know what should be the value of my pagination and I am not sure that if I can recursively send a Get request till all the data are fetched.

public JToken Search( ObjCode objcode, object parameters, int limit = 100 )
            {
            VerifySignedIn();
            string[] p = parameterObjectToStringArray(parameters, "sessionID=" + SessionID);
            JToken json = null;
            if (limit > 100)
            {
               json = client.DoGet(string.Format("/{0}/search?$$LIMIT={1}", objcode,limit),limit,p);
            }
            else
            {
              json = client.DoGet(string.Format("/{0}/search", objcode),limit,p);
            }

            return json;
            }


 public JToken DoGet(string path,int limit = 100 ,params string[] parameters)
        {
            return DoRequest(path,limit ,parameters);
        }

 public JToken DoRequest(string path,int limit, params string[] parameters)
        {
            if (!path.StartsWith("/"))
            {
                path = "/" + path;
            }
            string fullUrl = url + path + ToQueryString(parameters,limit);

            if (DebugUrls) Console.WriteLine("Requesting: {0}", fullUrl);

            WebRequest request = HttpWebRequest.CreateDefault(new Uri(fullUrl));
            using (WebResponse response = request.GetResponse())
            {
                using (Stream responseStream = response.GetResponseStream())
                {
                    return ReadResponse(responseStream);
                }
            }
        }

 private string ToQueryString(string[] parameters, int limit = 100)
    {
        StringBuilder sb = new StringBuilder();
        parameters.ToList().ForEach(s => sb.Append(s).Append("&"));
        if (sb.Length > 0)
        {
            sb.Remove(sb.Length - 1, 1);
        }
        return limit > 100 ? "&" + sb : "?" + sb;
    }

Answer 1

You will want to first call a COUNT to get a value for how many objects you are dealing with. This is done the same way as /search but using /count instead of search.

example

GET /attask/api/v4.0/proj/count?status=cur

you can then use $$FIRST to page through and pull your results.

your code would look comething like this

public JToken Search( ObjCode objcode, object parameters, int limit = 100 )
        {
        VerifySignedIn();
        string[] p = parameterObjectToStringArray(parameters, "sessionID=" + SessionID);
        JToken json = null; 
        JToken count = null;

        count = client.DoGet(string.Format("/{0}/count", objcode),limit,p);

        for(int i=0; i<count; i+limit){
            if (limit > 100)
            {
               json = client.DoGet(string.Format("/{0}/search?$$LIMIT={1}&$$FIRST={2}", objcode,limit,i),limit,p);
            }
            else
            {
               json = client.DoGet(string.Format("/{0}/search&$$FIRST={1}", objcode,i),limit,p);
            }

            return json;
        }

     }

this information can be found https://developers.workfront.com/api-docs/#PaginatedResponses