Paged load of resources in ruby

Question

I have a method that can only load 50 objects at once.

So I've come up with this piece of ruby code to keep loading until there are no more results:

objects = []
offset = 0
limit = 50

loop do 
  # Load paged objects using the current given offset
  new_objects = load_objects(some_url, limit: limit, start: offset)  
  offset += limit
  objects.concat(new_objects)

  # End the loop as soon as no more results are returned
  break if new_objects.count == 0
end 

Now while this works great, I was wondering if there is a more concise way to do this task in ruby.

Update: I am thinking of some collect-like approach like this:

# Pseudocode
objects = update_while_true([],0) do |result_array, limit|
    new_objects = load_objects(some_url, limit: 50, start: current)
    result_array.concat(new_objects)
    limit += 50

    # Should the loop be run again?
    new_objects.count > 0
end

Answer 1

I would extract it into a method, so your code would look like this:

limit = 50
objects = until_there_are_no_more_results do |offset|
  load_objects(some_url, limit: limit, start: offset)
end

The extracted method would contain very generic code, something like this (untested):

def until_there_are_no_more_results(&loader_proc)
  objects = []
  offset = 0

  loop do
    # Load paged objects using the current given offset
    new_objects = loader_proc.call(offset) # or: yield(offset)
    offset += limit
    objects.concat(new_objects)

    # End the loop as soon as no more results are returned
    break if new_objects.count == 0
  end

  objects
end

You could leave out the block parameter and use yield like below, but I prefer to have it in the method signature for clarity.

def until_there_are_no_more_results
  #...
    new_objects = yield(offset)