I'm writing an integration between my app and Salesforce, using the Salesforce REST api, specifically the query (SOQL) resource (https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_query.htm).
I need to be able to sync all contacts from Salesforce to my app, regardless of the number (e.g., there could be 10k, 100k, etc).
But I can't find a way to do this with the salesforce API. The query endpoint sets a max page size of 200 and max offset of 2000.
The documentation says that the response will include a nextRecordsUrl if there are more results than can be returned, but I don't understand how this can be used in this case. The API (jsforce npm library) requires a limit in the query, but even if I set the limit low (say 10 records), then the response includes just those 10, and no nextRecordsUrl (I assume because my request of 10 records is complete), and the limit can't be set higher than 200.
Is a different resource needed to get all objects?
Thanks.
I found the issue. I was using the FIELDS(ALL) function so that I don't have to explicitly list all of the fields in the SOQL query. The requirement to include a limit in the query is due to the use of this function:
https://newstechnologystuff.com/soql-fields-function/
If I specify field names explicitly, then there is no requirement to include a limit, and then, if the number of matching rows is greater than the batch size (default 2000), I get a nextRecordsUrl as expected.