how to deal with pagination when making api request using httparty rubygem

I'm trying to write a plugin for my Jekyll site which writes a sorted JSON response to a data file, but I'm having trouble dealing with the limit of items in the response. The basic response looks like this:

{
  "current_page": 1,
  "per_page": 50,
  "total_pages": 19,
  "url": "https://web.consonance.app/api/v2/products.json",
  "products": [
    {
      ...
      "pub_date": "2017-05-01"
      ...
    },
    ...
  ]
}

where products is an array of books that I want to sort by the date they were published (pub_date). There is no next_page url, only the current_page and total_pages. My plugin is very basic. It uses httparty to make the API request, then sorts the response as a hash by pub_date. So far, it looks like this:

require 'rubygems'
require 'httparty'
require 'json'

# http request to api

url = 'https://web.consonance.app/api/v2/products.json'
query = {
}
headers = {
  Authorization: "Token token=**************"
}
response = HTTParty.get(url, query: query, headers: headers)

# convert response to hash array

hash = JSON.parse(response.body)

# reverse sort product in products by date published

hash_sorted = hash['products'].sort_by! { |o| o['pub_date'] }.reverse!

# open _data file and write json response

File.open("./_data/products.json", "w") { |file| 
  file.puts JSON.pretty_generate(hash_sorted)
}

which gives me a JSON response of the first 50 items reverse sorted by pub_date. My question is:

How do I get the most recently published book (product) from the entire array, not just the first 50 results?

I have tried increasing the limit per page to its maximum, which is 500 items, but the total number of books spans more than one page, so I still have a pagination issue.

The available parameters for the request are documented here.

NB – I'm using httparty because I found it easy to make the request, but I am open to using other gems/methods. I've only just started learning Ruby, so please bear with me.

Solution

Using current_page and total_pages as conditions, I looped through the pages of the response and assigned all products to an array:

while current_page <= total_pages do

  url = 'https://web.consonance.app/api/v2/products.json'
  query = {
    'page' => "#{current_page}",
  }
  request = HTTParty.get(url, query: query)

  hash = JSON.parse(request.body)

  hash['products'].each do |item|
    product_array.push(item)
  end

  current_page += 1

end