Search code examples
nltkwordnetlemmatization

Is it possible to speed up Wordnet Lemmatizer?

I'm using the Wordnet Lemmatizer via NLTK on the Brown Corpus (to determine if the nouns in it are used more in their singular form or their plural form).
i.e. from nltk.stem.wordnet import WordNetLemmatizer
l = WordnetLemmatizer()

I've noticed that even the simplest queries such as the one below takes quite a long time (at least a second or two).
l("cats")

Presumably this is because a web connection must be made to Wordnet for each query?..
I'm wondering if there is a way to still use the Wordnet Lemmatizer but have it perform much faster? For instance, would it help at all for me to download Wordnet on to my machine? Or any other suggestions?

I'm trying to figure out if the Wordnet Lemmatizer can be made faster rather than trying a different lemmatizer, because I've found it works the best among others like Porter and Lancaster.

Solution

  • I've used the lemmatizer like this

    from nltk.stem.wordnet import WordNetLemmatizer # to download corpora: python -m    nltk.downloader all
lmtzr = WordNetLemmatizer() # create a lemmatizer object
lemma = lmtzr.lemmatize('cats')

    It is not slow at all on my machine. There is no need to connect to the web to do this.