Search code examples
pythonpython-3.xgoogle-apipython-newspaper

How to take output from iterating, store that in a dictionary


So I have this script (running Python 3.5) using Google API and Newspaper. It searches google for articles that have to do with sleep. And then using Newspaper, I iterate over those URLS. And all I'm asking Newspaper to do is return a list of keywords of that article, which I call by writing article.keywords .

for url in google.search('sleep', num=2, stop=1):
    article = Article(url)      
    article.download() 
    article.parse()
    article.nlp()     
    print(article.keywords)

The keywords that are returned (for a given article) look like this:

['education', 'nights', 'start', 'pill', 'supplement', 'research', 'national', 'sleep', 'sleeping', 'trouble', 'using', 'taking']

But I want to create a dictionary full of ALL of the keywords for all the results: That is, the keywords for each article that is being iterated over. How would I do that?


Solution

  • Assuming the dictionary key should be an article url:

    keywords = {}
    for url in google.search('sleep', num=2, stop=1):
        article = Article(url)      
        article.download() 
        article.parse()
        article.nlp()  
    
        keywords[url] = article.keywords
    
    print(keywords)
    

    Or, if you want to have a list of all the keywords from all the articles:

    keywords = []
    for url in google.search('sleep', num=2, stop=1):
        article = Article(url)      
        article.download() 
        article.parse()
        article.nlp()  
    
        keywords += article.keywords
    
    print(keywords)