Search code examples
pythonnlpword2vec

How to use Word2Vec with two inputs in a loop?


I'm trying to create a similarity between two words using word2vec, I was successful, while doing it manually. but I have two big txt files. I want to create a loop. I tried a couple methods for looping but I was unsuccessful. so I decided to ask expert.

my code :

import gensim

model = gensim.models.Word2Vec.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True)
with open('myfile1.txt', 'r') as f:
    data1 = f.readlines()

with open('myfile2.txt', 'r') as f:
    data2 = f.readlines()

data = zip(data1, data2)

with open('myoutput.txt', 'a') as f:
    for x in data: 
        output = model.similarity(x[1], x[0])  # reading each word form each files
        out = '{} : {} : {}\n'.format(x[0].strip(), x[1].strip(),output)  
        f.write(out)

my input1, (text1)

street 
spain 
ice
man

my input2 (text2)

florist
paris 
cold 
kid

I want this output (output.txt)

street florist 0.19991447551502498
spain paris 0.5380033328157873
ice cold 0.40968857572410483
man kid  0.42953233870042506

Solution

  • import gensim
    
    model = gensim.models.Word2Vec.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True)
    
     file1 = []
     file2 = []
    
     with open('myfile1.txt','rU') as f:
     for line in f:
     file1.append(line.rstrip())
    
     with open('myfile2.txt','rU') as f1:
     for line1 in f1:   
     file2.append(line1.rstrip())
    
     resutl=[]
     f=open('Output2.txt', "w") 
     for i in file1  :
     for g  in file2 :
            temp=[]
            temp.append(i)
            temp.append(g)
            w = model.similarity(i,g)
            temp.append(w)
            result=i+','+g+','+str(w)
    
            f.write(result)
            f.write('\n')
    
            f.close()
    

    You had problem with the loop, the two loops should be together.