Search code examples
pythonvisual-studio-codenlpgensim

How to import gensim summarize

I got gensim to work in Google Collab by following this process:

!pip install gensim
from gensim.summarization import summarize

Then I was able to call summarize(some_text)

Now I'm trying to run the same thing in VS code:

I've installed gensim: pip3 install gensim

but when I run

from gensim.summarization import summarize

I get the error

Import "gensim.summarization" could not be resolvedPylancereportMissingImports

I've also tried from gensim.summarization.summarizer import summarize with same error. Regardless I haven't been able to call the function summarize(some_text) outside of Google Collab.

Solution

  • The summarization code was removed from Gensim 4.0. See:

    https://github.com/RaRe-Technologies/gensim/wiki/Migrating-from-Gensim-3.x-to-4#12-removed-gensimsummarization

    12. Removed gensim.summarization

    Despite its general-sounding name, the module will not satisfy the majority of use cases in production and is likely to waste people's time. See this Github ticket for more motivation behind this.

    If you need it, you could try:

    • installing an older gensim version (such as 3.8.3, the last official release in which it remained); or…
    • copy the source code out to your own local module

    However, I expect you'd likely be disappointed by its inflexibility and how little it can do.

    It was only extractive summarization - choosing a few key sentences from those that already exist. That only gives impressive results when the source text was already well-written in an expository style mixing high-level overview sentences with separate detail sentences. And, its method of analyzing/ranking words was very crude & hard-to-customize – totally unconnected to the more generic/configurable/swappable approaches used elsewhere in Gensim or in other text libraries.