Search code examples
phprssaccess-denied

How to avoid denial of service (DoS) reading Google News RSS feeds

I have over 1000 clients and I developed a simple PHP script to loop through their names searching for any news using Google News RSS.

I loop this string substituting $searchString for the relevant name:

$GoogleSearch = "https://news.google.com/news/feeds?q=".$searchString."&output=rss";

I am also using a "call_user_func" in an attempt to control (serialize) the requests to Google (i.e. a "delegate method").

Trouble is Google is too smart for their own good and they detected my IP address sending these requests and they blocked my access!

Any suggestions how I can avoid a DoS?

Thanks.

Solution

  • You probably just need to slow down the requests you're sending to google. Either put pauses in your scripts or separate it so that it runs at different times for different clients. For example, if you have 1000 clients, you could run one request every 3.6 seconds and you'd have all of your clients updated within an hour.