Updating page number on API call with recursion

I'm making API calls and the service I am using only returns a certain number of results per page(I need all the results, not just first page). Fortunately they have a key on the JSON object called "total_pages" that says how many pages there are. There's also a "current_page" attribute. You can add a pager to the end of the URL you are calling like so: "example.com/endpoint&page=2"

I realized that the only way to get all of the pages is to loop through the total_pages number and somehow create another call to the api with the new pager info.

This can be done with a for or while loop by incrementing the current_page attribute but I can't seem to figure out how to do it with recursion. I have come up with a way to do it with Agent that holds and updates the current_page state but it seems unnecessary.

Agent.start(fn -> 1 end, name: current_page)
Agent.update(:current_page, fn curr_page -> state + 1 end)

Is there a way to do this in Elixir, maybe with a cond statement that checks if the current_page == total_pages and if not it runs a function that calls the API again?

THanks

Solution

Yes, an Agent is overkill for this. A simple recursive function can solve this elegantly. Here's a basic implementation that should help you get started:

defmodule A do
  def fetch(current_page) do
    response = get("/?page=#{current_page}")
    if response.total_pages == current_page do
      :ok
    else
      fetch(current_page + 1)
    end
  end

  # Sample function that acts like an HTTP GET request.
  def get(url) do
    IO.puts "GET #{url}"
    %{total_pages: 5}
  end
end

A.fetch(1)

Output:

GET /?page=1
GET /?page=2
GET /?page=3
GET /?page=4
GET /?page=5

We start with page 1. After fetching the contents of the page, we check if the current page equals the total_pages in the response. If it does, we stop, otherwise we recurse with current_page set to current_page + 1.