I'm trying to count the occurences of multiple keywords within each phrases of a dataframe. This seems similar to other questions but not quite the same.
Here we have a df and a list of lists containing keywords/topics:
df=pd.DataFrame({'phrases':['very expensive meal near city center','very good meal and waiters','nice restaurant near center and public transport']})
topics=[['expensive','city'],['good','waiters'],['center','transport']]
for each phrase, we want to count how many words match in each separate topic. So the first phrase should score 2 for 1st topic, 0 for 2nd topic and 1 for 3rd topic, etc
I've tried this but it does not work:
from collections import Counter
topnum=0
for t in topics:
counts=[]
topnum+=1
results = Counter()
for line in df['phrases']:
for c in line.split(' '):
results[c] = t.count(c)
counts.append(sum(results.values()))
df['topic_'+str(topnum)] = counts
I'm not sure what i'm doing wrong, ideally i would end up with a count of matching words for each topic/phrases combinations but instead the counts seem to repeat themselves:
phrases topic_1 topic_2 topic_3
very expensive meal near city centre 2 0 0
very good meal and waiters 2 2 0
nice restaurant near center and public transport 2 2 2
Many thanks to whoever can help me. Best Wishes
Here is a solution that defines a helper function called find_count and applies it as a lambda to the dataframe.
import pandas as pd
df=pd.DataFrame({'phrases':['very expensive meal near city center','very good meal and waiters','nice restaurant near center and public transport']})
topics=[['expensive','city'],['good','waiters'],['center','transport']]
def find_count(row, topics_index):
count = 0
word_list = row['phrases'].split()
for word in word_list:
if word in topics[topics_index]:
count+=1
return count
df['Topic 1'] = df.apply(lambda row:find_count(row,0), axis=1)
df['Topic 2'] = df.apply(lambda row:find_count(row,1), axis=1)
df['Topic 3'] = df.apply(lambda row:find_count(row,2), axis=1)
print(df)
#Output
phrases Topic 1 Topic 2 Topic 3
0 very expensive meal near city center 2 0 1
1 very good meal and waiters 0 2 0
2 nice restaurant near center and public transport 0 0 2