Search code examples
pythonsearch-enginelimit

problem with python xgoogle library. How to increase the limit of 10 results?

I use http://www.catonmat.net/blog/python-library-for-google-search/ to make queries at Google search engine but the number of results I get is limited to 10. I used the results_per_page property and I set it to 50, 100 etc. but the number of results didn't changed. Is there a way to get more results? Is there another python lib without these constraints for Google searching?

Thank you in advance, Thanasis

Solution

  • You will need to loop through the pages: listed below is an example; remember to place a thread sleep between each call to Google: otherwise your IP will be blocked if your search is large (getting it unblocked is a real hassle and your ISP may get on your case).

    from xgoogle.search import GoogleSearch, SearchError
try:
  page = 1
  gs = GoogleSearch("foo bar")  
  gs.results_per_page = 100
  results = []
  while page < 10:
      gs.page = page
      results += gs.get_results()
      page += 1
except SearchError, e:
  print "Search failed: %s" % e
for res in results:
    print res.url