Search code examples
python-3.xcsvfilereadergoogle-search

Is there a way to fetch the url from google search result when a csv file full of keyword is uploaded in Python?

Is it possible to obtain the url from Google search result page, given the keyword? Actually, I have a csv file that contains a lot of companies name. And I want there website which shows up on the top of search result in google, when I upload that csv file it fetch the company name/keyword and put it on the search field.

For eg: - stack overflow, this is one of the entry in my csv file and it should be fetched and put in the search field, and it should return the best match/first url from search result. Eg: - www.stackoverflow.com

And this returned result should be stored in the same file which I have uploaded and next to the keyword for it searched.

I am not aware much about these concepts, so any help will be very appreciated. Thanks!

Solution

  • google package has one dependency on beautifulsoup which need to be installed first.

    then install : pip install google

    search(query, tld='com', lang='en', num=10, start=0, stop=None, pause=2.0)

    query : query string that we want to search for.

    tld : tld stands for top level domain which means we want to search our result on google.com or google.in or some other domain.

    lang : lang stands for language.

    num : Number of results we want.

    start : First result to retrieve.

    stop : Last result to retrieve. Use None to keep searching forever.

    pause : Lapse to wait between HTTP requests. Lapse too short may cause Google to block your IP. Keeping significant lapse will make your program slow but its safe and better option.

    Return : Generator (iterator) that yields found URLs. If the stop parameter is None the iterator will loop forever.

    Below code is the solution for your question.

    import pandas
from googlesearch import search 

df = pandas.read_csv('test.csv')

result = []
for i in range(len(df['keys'])):
    for j in search(df['keys'][i], tld="com", num=10, stop=1, pause=2): 
        result.append(j)

dict1 = {'keys': df['keys'], 'url': result}  
df = pandas.DataFrame(dict1) 
df.to_csv('test.csv')

    Sample input format file image:

    Sample input format file

    Output File Image:

    enter image description here