Search code examples
python-3.xweb-scrapinggoogle-search

Searching using google and python and storing the first link?


I am trying to make use of google search and get the first URL from the search results. I tried to make use of the google custom search api. But it seems over the top for such a simple task. Hence, I am trying to use this interesting package I found: https://pypi.python.org/pypi/google

This is what I came up with so far

from google import search
url = search('my search entry', stop=1)
for result in url:
    print(url)

It seems that search() returns several generator objects. This is my return:

<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>
<generator object search at 0x10e230048>

However, I want the first external url/link. Is there a way to do that? I tried list() - but the generator seems empty.


Solution

  • From http://pythonhosted.org/google/ , the signature of search is

    generator search(query, tld='com', lang='en', num=10, start=0, stop=None, pause=2.0)
    

    Try setting num = 1 and stop = 0.