Search code examples
pythonstringpandasword2vecgensim

How to write words having similarity above .6 to a specific word from a dictionary to a dataframe in pandas


I have a word2vec dictionary which has a list of similar words to given word.

Example

model.most_similar("ltd")
[('limited', 0.7886955142021179),
 ('limi', 0.6512018442153931),
 ('limite', 0.6031635999679565),
 ('wilford', 0.5938706994056702),
 ('lt', 0.583463728427887),
 ('lighttech', 0.5828145146369934),
 ('rmc', 0.5821658372879028),
 ('tomoike', 0.5752800703048706),
 ('jd', 0.5751883387565613),
 ('nxp', 0.5725069046020508)]

I want to create dataframe containing root and similar_words(having similarity above .6)

Currently I am able to write all the similar words corresponding to root word

words = y
similar = [[item[0] for item in model.most_similar(word)[:6]] for word in words]
similarity_matrix = pd.DataFrame({'Root_Word': words, 'Similar_Words': similar})

Current Output

Root_Word    Similar_word
[st]         [st., sreet, rd;, yop, tseun, tsven] 
[limited]    [ltd, lt, wt, serial, (h.k., dk] 
[centre]     [cent, ct, cte, entre, ctr., ce]

Expected output is have only Similar words which having similarity above .6.

How can this be done


Solution

  • Based on your current method:

     similar = [[item[0] for item in model.most_similar(word) if item[1] > 0.6] for word in words]