I'm having some trouble with sending batch requests for the pagespeed API. From what I have seen in my google developers console I should have 100 requests per second with a max of 25,000 per day. However, I'm running into a problem even just trying to do 50 per second. I have tried to impose timing into my ruby application and for some reason that doesn't change the error. I still get the rateLimitExceeded Error from google on a decent amount of the results. I'm doing this on arrays containing ~1000 urls if that matters.
Here's my batch function, I call it in a loop from another function, I thought this might work better for timing. But it didn't seem to change anything.
def send_request(urls)
@psservice.batch do |ps|
urls.each do |url|
ps.run_pagespeed(url) do |result, err|
err.nil? ? @data.push(result) : @errors.push("#{url}, #{err}")
end
end
end
end
This gets called from
@urls.each_slice(50).to_a.each do |url_list|
send_request(url_list, options)
sleep(1)
end
Any ideas why this would occur? Thanks in advance
I misread the quota for Google Pagespeed - it's 100 requests / 100 seconds. So one request a second. It seems the more I check that it actually is 1 request / second. I still have trouble trying to run 100 requests then waiting 100 seconds.