Searching in Google with Python

Question

I want to search a text in Google using a python script and return the name, description and URL for each result. I'm currently using this code:

from google import search

ip=raw_input("What would you like to search for? ")

for url in search(ip, stop=20):
     print(url)

This returns only the URL's. How can I return the name and description for each URL?

Answer 1

Not exactly what I was looking for, but I found myself a nice solution for now (I might edit this if I will able to make this better). I combined searching in Google like I did (returning only URL) and the Beautiful Soup package for parsing HTML pages:

from googlesearch import search
import urllib
from bs4 import BeautifulSoup

def google_scrape(url):
    thepage = urllib.urlopen(url)
    soup = BeautifulSoup(thepage, "html.parser")
    return soup.title.text

i = 1
query = 'search this'
for url in search(query, stop=10):
    a = google_scrape(url)
    print str(i) + ". " + a
    print url
    print " "
    i += 1

This gives me a list of the title of pages and the link.

And another great solutions:

from googlesearch import search
import requests

for url in search(ip, stop=10):
            r = requests.get(url)
            title = everything_between(r.text, '<title>', '</title>')