Search code examples
ruby-on-railsapihttppaginationzendesk

Consuming paginated resources using HTTP with Ruby on Rails


I'm building out a platform for displaying data in charts that pulls from Zendesk's API. I'm running into trouble in that only 100 records at a time can be pulled with one call. How do I pull multiple pages of records from this resource?

Here is the code I use to make the call:

require 'net/http'
require 'uri'
require 'json'

#imports User data from the zendesk api and populates the database with it. 

uri = URI.parse("https://samplesupport.zendesk.com/api/v2/users.json")
request = Net::HTTP::Get.new(uri)
request.content_type = "application/json"
request.basic_auth("[email protected]", "samplepass")

req_options = {
  use_ssl: uri.scheme == "https",
}

@response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
  http.request(request)
end
puts @response.body
puts @response.message
puts @response.code

This works fine for calling down one 'page' of resources...any help with grabbing multiple pages using my script would be greatly appreciated. Thank you!


Solution

  • Based on ZenDesk's documentation they return a next_page attribute in their payload. So you should just check for its existence and then query again if it exists. Repeat as needed.

    require 'json'
    # setup to query for the first page
    results = JSON.parse(@response.body)
    
    users = results['users'] #to get the users
    if results['next_page']
      # Do another query to results['next_page'] URL and add to users list