gensim lemmatize error generator raised StopIteration

Question

I'm trying to execute simple code to lemmatize string, but there's an error about iteration. I have found some solutions which are about reinstalling web.py, but this not worked for me.

python code

from gensim.utils import lemmatize
lemmatize("gone")

error is

---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
I:\Anaconda\lib\site-packages\pattern\text\__init__.py in _read(path, encoding, comment)
    608             yield line
--> 609     raise StopIteration
    610 

StopIteration: 

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
<ipython-input-4-9daceee1900f> in <module>
      1 from gensim.utils import lemmatize
----> 2 lemmatize("gone")

-------------------------------------------------------------------------------------

I:\Anaconda\lib\site-packages\pattern\text\__init__.py in <genexpr>(.0)
    623     def load(self):
    624         # Arnold NNP x
--> 625         dict.update(self, (x.split(" ")[:2] for x in _read(self._path) if len(x.split(" ")) > 1))
    626 
    627 #--- FREQUENCY -------------------------------------------------------------------------------------

RuntimeError: generator raised StopIteration

Answer 1

The error message is misleading – it occurs when there's nothing to properly lemmatize.

By default, lemmatize() only accepts word tags NN|VB|JJ|RB. Pass in a regexp that matches any string to change this:

>>> import re
>>> lemmatize("gone", allowed_tags=re.compile('.*'))
[b'go/VB']