Search code examples
rubygoogle-apipagespeed

Ensure Batch request for Google API occurs once per second


I'm having some trouble with sending batch requests for the pagespeed API. From what I have seen in my google developers console I should have 100 requests per second with a max of 25,000 per day. However, I'm running into a problem even just trying to do 50 per second. I have tried to impose timing into my ruby application and for some reason that doesn't change the error. I still get the rateLimitExceeded Error from google on a decent amount of the results. I'm doing this on arrays containing ~1000 urls if that matters.

Here's my batch function, I call it in a loop from another function, I thought this might work better for timing. But it didn't seem to change anything.

def send_request(urls)
  @psservice.batch do |ps|
    urls.each do |url|
      ps.run_pagespeed(url) do |result, err|
        err.nil? ? @data.push(result) : @errors.push("#{url}, #{err}")
      end
    end
  end
end

This gets called from

@urls.each_slice(50).to_a.each do |url_list|
  send_request(url_list, options)
  sleep(1)
end

Any ideas why this would occur? Thanks in advance


Solution

  • I misread the quota for Google Pagespeed - it's 100 requests / 100 seconds. So one request a second. It seems the more I check that it actually is 1 request / second. I still have trouble trying to run 100 requests then waiting 100 seconds.