Search code examples
web-scrapinggoogle-search

Possible to show and store top 3 results of Google Search?


Is it possible to show and store the results obtained from a Google Search query? The search is performed from the commons-http client, through my webapp. My questions are:

  1. Is it permissible?
  2. Is it ethical?
  3. Is it possible?

I have heard that google changes tags and blocks scrapers. Is it correct? Any other way to do it?


Solution

  • It is certainly possible to scrape Google for results - many do. You can use proxies to mask your IP and avoid being blocked.

    Make sure you set your user-agent to a browser.

    Google doesn't like this and provides a search API, but it is pretty useless.