I'm trying to figure out how to query a resource to see how many rows it has before I request the entire resource at once, or whether I use paging to bring back rows in batches.
For example, this resource here:
https://data.cityofnewyork.us/Transportation/Bicycle-Routes/7vsa-caz7
1) In cases where I know the number of rows, I can use the $limit parameter to ensure I get back everything. For example, this dataset has about 17,000 rows, so giving it a $limit of 20000 gets all of them.
For example:
https://data.cityofnewyork.us/resource/cc5c-sm6z.geojson?$limit=20000
also...
2) I thought maybe to make a metadata call, but while this request here returns metadata, number of rows is not part of it:
https://data.cityofnewyork.us/api/views/metadata/v1/cc5c-sm6z
However, I would like to know how many rows are in the dataset before I decide how to request them: all at once with the $limit parameter, or paging with the $limit and $offset parameters.
Ideas?
One method could be to count the rows using the COUNT function in the API call.
Should note that YMMV on this approach. Generally, the maximum is around 50,000 rows before you need to switch to paging. Generally, I'll always throw a 50k limit and have paging ready if it's larger.