API times out due to limit and offset

Question

I'm trying to pull items from one of our client's Podio instance. It has around 35k records, so I'm setting a limit and offset of 500

loop do
  current_offset_value = offset.next
  puts "LIMIT: #{LIMIT}, OFFSET: #{current_offset_value}"
  Podio::Item.find_by_filter_id(app_id, view_id, limit: LIMIT, remember: true, offset: offset.next).all.each do |item|
      yield item
  end
end

However, the code just hangs after the first two calls and returns a timeout error

LIMIT: 500, OFFSET: 0
LIMIT: 500, OFFSET: 500
creates a CSV file from a table (FAILED - 1)

Failures:

SourceTableSync::LocalCsvDumper::CitrixPodio creates a CSV file from a table
 Failure/Error:
   Podio::Item.find_by_filter_id(app_id, view_id, limit: LIMIT, remember: true, offset: current_offset_value).all.each do |item|
     yield item
   end

 Faraday::TimeoutError:
   Net::ReadTimeout with #<TCPSocket:(closed)>

Answer 1

Looks like Podio API is timing out while getting 500 items, probably your items are big or has some relationship to other apps and it simply takes too much time to fetch it all.

I'd try some smaller number (e.g. 100 or 200) to see if that will work :)