I want to find out the trigrams of a corpus but with the restriction that at least two words of the trigrams are not proper nouns. This is my code so far.
def collocation_finder(text,window_size):
ign = stopwords.words('english')
#Clean the text
finder = TrigramCollocationFinder.from_words(text, window_size)
finder.apply_freq_filter(2)
finder.apply_word_filter(lambda w: len(w) < 2 or w.lower() in ign)
finder.apply_word_filter(lambda w: next(iter(w)) in propernouns)
trig_mes = TrigramAssocMeasures()
#Get trigrams based on raw frequency
collocs = finder.nbest(trig_mes.raw_freq,10)
scores = finder.score_ngrams( trig_mes.raw_freq)
return(collocs)
Where propernouns is a list of all the proper nouns in the corpus.
The thing is that my last word filter the one that was supposed to make sure that I don't go over my restriction. Any ideas?
This should be want you want
finder.apply_ngram_filter(lambda w1, w2, w3: sum([w1 n propernouns, w2 in propernouns, w3 in propernouns]) >= 2)