Search code examples
pythongoogle-search-api

Google search on browser and google search via the custom search api give different results for the same query


I have a python program that takes the md5 & sha1 hash values of passwords and searches for them on the internet using Google's custom search api. The problem is that I'm getting 0 results(which means the hash probably isn't in a rainbow table) when I run the program. But when I searched using my browser, I get a whole bunch of results, in fact at least 10 pages of results.

Could the problem lie in the cx value I used? I picked it up from the sample program provided by google as I couldn't figure out how to get one for myself. Or does the custom search api give only selected results and it's futile trying to get more results from it?


Solution

  • The Google Custom Search API let's you search the Google indexes for a specific website only, and you will not find any results from anywhere else on the internet. The cx parameter tells Google what website you want to search.

    From the Google Custom Search Engine page:

    With Google Custom Search, add a search box to your homepage to help people find what they need on your website.

    You could use the deprecated Google Web Search API (JavaScript API, should work until November 2013), or you'd have to scrape the HTML UI provided to your browser instead (also see What are the alternatives now that the Google web search API has been deprecated?).