python - pandas: creating a new dataframe with a list of keywords

Question

I have a dataframe of news headlines, like this:

df = pd.DataFrame({'texto': ['El Hubble registró imágenes de una "mariposa interestelar"',
                             'Con su Audi atropelló y mató a una pareja: lo juzgan y podrían darle 25 años',
                             'Clarín cumplió 70 años y lo celebró con su personal',
                             'A diez años de Katrina, Obama visitó Nueva Orleans y celebró la recuperación',
                             'La oposición volvió a juntarse y pidió la boleta única electrónica en octubre',
                             'Hallan a casi 50 “sin papeles” muertos en un camión en Austria'],
                   'id': [1,2,3,4,5,6]})

df['texto'] = df['texto'].str.lower()

searchwords = ['hubble','audi','obama','clarín','austria']

And I am trying to search if each and every single of some keywords selected appear in above dataframe to create a new one. I have tried 'startswith', but only contains the sentences where the keyowrds are first.

Code:
df = df[df['texto'].str.startswith(tuple(searchwords))]

Output:

clarín cumplió 70 años y lo celebró con su per...   3

Do you know a way to create a pandas line or a function to create a filter like this?

Thanks!

Answer 1

Use

df2 = df[df['texto'].str.contains('|'.join(searchwords))]

Explanation:

'|'.join(searchwords) produces a string with the search words separated by the | (OR operator) symbol. df['texto'].str provides each element of the column as a string and this is then checked to see if it contains any of the search words (hence the OR operator). This produces a series of True / False values. The boolean series is then used to select the rows from the DataFrame df which meet the True condition and these form the new DataFrame df2.