Search code examples
pythonnltkwordnet

replaceWithHyponym function selects a word from the text and replaces it with a hyponym, resulting in index out of range


self.lines.append([random.randint(10,700),random.randint(10,950), \
                   "Once there was a dog and a cat","l0"])
self.lines.append([random.randint(10,700),random.randint(10,950), \
                   "Who decided to have an extended natter","l1"])

This is the text analyzed.

def replaceWithHyponym(lines):
        ll = lines.pop(random.randrange(len(lines)))
        tokens = nltk.word_tokenize(ll[2])
        wordIdx = random.randrange(len(tokens))
        wordToBeReplaced = tokens[wordIdx]

        print('wordToBeReplaced')
        print(wordToBeReplaced)
        
        vehicle = wordnet.synsets(wordToBeReplaced)[0]
        typeofword = wordnet.synsets(vehicle.name())
        typesOfhyponyms = list(set([w for s in vehicle.closure(lambda s: s.hyponyms()) for w in s.lemma_names()]))
        print('typesOfhyponyms')            
        print(typesOfhyponyms)

        if typesOfhyponyms:
            tokens[wordIdx] = random.choice(typesOfhyponyms)
        else:
            tokens[wordIdx] = wordToBeReplaced
        print('tokens')
        print(tokens)

        newLine = " ".join(tokens)
        lines.append([ll[0],ll[1],newLine,ll[3]])
        return lines

This function runs continously until the text is poetic enough.

Error:

*** in run

[[325, 130, 'Once there was a dog and a cat', 'l0'], [684, 268, 'Who decided to have an extended natter', 'l1']]

2

executing  <function replaceWithHypernym at 0x000001BFA0142708>

wordToBeReplaced:

a

typesOfhyponyms:

[]

tokens:

['Once', 'there', 'was', 'a', 'dog', 'and', 'a', 'cat']

*** in run

[[684, 268, 'Who decided to have an extended natter', 'l1'], [325, 130, 'Once there was a dog and a cat', 'l0']]

2
executing  <function replaceWithHypernym at 0x000001BFA0142708>

wordToBeReplaced:

decided

typesOfhyponyms:

['calliper', 'adjudicate', 'reverse', 'stick', 'vote', 'pick_out', 'dedicate', 'define', 'overthrow', 'cream_off', 'turn_thumbs_down', 'put_up', 'regularise', 'anoint', 'calibrate', 'sieve', 'standardise', 'standardize', 'override', 'will', 'redefine', 'orientate', 'shoot', 'determine', 'measure_out', 'sieve_out', 'mensurate', 'pick_over', 'take_out', 'empanel', 'propose', 'judge', 'panel', 'quantify', 'set', 'cull_out', 'field', 'vote_in', 'reset', 'overturn', 'sift', 'orient', 'choose', 'poll', 'guide_on', 'screen_out', 'order', 'adopt', 'ballot', 'reelect', 'excerpt', 'overrule', 'rule_in', 'rule', 'outvote', 'decree', 'espouse', 'winnow', 'set_apart', 'co-opt', 'select', 'regularize', 'vote_down', 'resolve', 'limit', 'regulate', 'govern', 'impanel', 'skim_off', 'assign', 'think_of', 'hand-pick', 'pick', 'rule_out', 'seal', 'return', 'adhere', 'measure', 'detail', 'gauge', 'elect', 'screen', 'name', 'caliper', 'guide', 'purpose', 'draw', 'reorientate', 'put_forward', 'fix', 'take', 'extract', 'triangulate', 'dial', 'follow', 'specify', 'reorient', 'court-martial', 'plump', 'single_out', 'nominate', 'zone', 'try', 'district', 'sort', 'write_in', 'go']

tokens:

['Who', 'overthrow', 'to', 'have', 'an', 'extended', 'natter']

*** in run

[[325, 130, 'Once there was a dog and a cat', 'l0'], [684, 268, 'Who overthrow to have an extended natter', 'l1']]

2

executing  <function replaceWithHypernym at 0x000001BFA0142708>

wordToBeReplaced:

a

typesOfhyponyms:

[]

tokens:

['Once', 'there', 'was', 'a', 'dog', 'and', 'a', 'cat']

*** in run

[[684, 268, 'Who overthrow to have an extended natter', 'l1'], [325, 130, 'Once there was a dog and a cat', 'l0']]

2

executing  <function replaceWithHypernym at 0x000001BFA0142708>

wordToBeReplaced:

a

typesOfhyponyms:

[]

tokens:

['Once', 'there', 'was', 'a', 'dog', 'and', 'a', 'cat']

*** in run

[[684, 268, 'Who overthrow to have an extended natter', 'l1'], [325, 130, 'Once there was a dog and a cat', 'l0']]

2

executing  <function replaceWithHypernym at 0x000001BFA0142708>

wordToBeReplaced:

there

typesOfhyponyms:

[]

tokens:
['Once', 'there', 'was', 'a', 'dog', 'and', 'a', 'cat']

*** in run

[[684, 268, 'Who overthrow to have an extended natter', 'l1'], [325, 130, 'Once there was a dog 
and a cat', 'l0']]

2

executing  <function replaceWithHypernym at 0x000001BFA0142708>

wordToBeReplaced:

extended

typesOfhyponyms:

['territorialise', 'stretch', 'territorialize', 'globalize', 'globalise', 'broaden']

tokens:

['Who', 'overthrow', 'to', 'have', 'an', 'broaden', 'natter']

*** in run

[[325, 130, 'Once there was a dog and a cat', 'l0'], [684, 268, 'Who overthrow to have an broaden natter', 'l1']]

2

executing  <function replaceWithHypernym at 0x000001BFA0142708>

wordToBeReplaced:

to

***Exception in Tkinter callback***

***Traceback (most recent call last):***

  ***File "C:\Users\essam\Anaconda3\lib\tkinter\__init__.py", line 1705, in __call__***
    ***return self.func(*args)***

  ***File "C:\Users\essam\Anaconda3\lib\tkinter\__init__.py", line 749, in callit***
    ***func(*args)***

  ***File "<ipython-input-145-8131f290a692>", line 25, in run***
   ***self.lines = ag(self.lines)***

  ***File "<ipython-input-145-8131f290a692>", line 45, in replaceWithHypernym***
    ***vehicle = wordnet.synsets(wordToBeReplaced)[0]***

***IndexError: list index out of range***

Solution

  • The error is coming from vehicle = wordnet.synsets("to")[0] (if I've understood the debug output correctly).

    The error says [0] is out of range, which must mean wordnet.synsets("to") has returned an empty list. I.e. "to" is not found in any synsets.

    So you need to assign wordnet.synsets(wordToBeReplaced) to a variable, and check the list length is at least 1 (and return early if it is zero?) before getting [0].

    (If you return early when no synsets are found, just be aware of the potential for infinite loops. I'd be inclined to return false, or throw an exception, to indicate that no words were changed.)