I’m trying to use GLEU score from NLTK for quality evaluation of the machine translation. I wanted to check this code with two identical sentences, I’m comparing two sentences and not corpuses. But as a result I’m getting 0.015151515151515152. What am I doing wrong? Two identical sentences should get score 1.0.
My code:
from nltk.translate.gleu_score import sentence_gleu
hyp1 = ['It', 'is', 'a', 'guide', 'to', 'action', 'which', 'ensures', 'that', 'the', 'military', 'always', 'obeys', 'the', 'commands', 'of', 'the', 'party']
ref1a = ['It', 'is', 'a', 'guide', 'to', 'action', 'which', 'ensures', 'that', 'the', 'military', 'always', 'obeys', 'the', 'commands', 'of', 'the', 'party']
gleu_score = sentence_gleu(ref1a, hyp1)
print(gleu_score)
My result:
0.015151515151515152
Process finished with exit code 0
Am I mistaken? Please help!
The first parameter to sentence_gleu should be a list of lists (a list of reference sentences, where each sentence is a list of words).
Try calling it like this:
gleu_score = sentence_gleu([ref1a], hyp1)