Search code examples
pythonnlpbert-language-modelsummarize

BERT Summarization for a column of texts

I'm trying to summarize 100 products descriptions from a dataset. To do so i simply tried to use summarizers

!pip install summarizers -q

from summarizers import Summarizers 

import pandas as pd

It worked pretty well for one sentence at a time.

textpanasonic="The NN-CS89L offers next-level cooking convenience. Its four distinct cooking methods - steaming, baking, grilling and microwaving ensure your meals are cooked or reheated to perfection. Its multi-function capabilities can be combined to save time without compromising taste, texture or nutritional value. It’s the all-in-one kitchen companion designed for people with a busy lifestyle."



summ(textpanasonic)

The NN-CS89L offers next-level cooking convenience.

but do you know if it is possible to create a new column with a summary for each comment ?

ValueError: text input must of type str (single example), List[str] (batch or single pretokenized example) or List[List[str]] (batch of pretokenized examples).

Thank you in advance ^^

Solution

  • You can simply apply summ to the column with summaries. Since summ can take a list as input it will also process a pandas Series. To provide an example with multiple rows:

    import pandas as pd
from summarizers import Summarizers
summ = Summarizers()

data = ["The NN-CS89L offers next-level cooking convenience. Its four distinct cooking methods - steaming, baking, grilling and microwaving ensure your meals are cooked or reheated to perfection. Its multi-function capabilities can be combined to save time without compromising taste, texture or nutritional value. It’s the all-in-one kitchen companion designed for people with a busy lifestyle.", "These slim and stylish bodies are packed with high performance. The attractive compact designs and energy-saving functions help Panasonic Blu-ray products consume as little power as possible. You can experience great movie quality with this ultra-fast booting DMP-BD89 Full HD Blu-ray disc player. After starting the player, the time it takes from launching the menu to playing a disc is much shorter than in conventional models. The BD89 also allows for smart home networking (DLNA) and provides access to video on demand, so that home entertainment is more intuitive, more comfortable, and lots more fun."]
df = pd.DataFrame(data, columns=['summaries'])
df['abstracts'] = df['summaries'].apply(summ)
    summaries abstracts
    0 The NN-CS89L offers next-level cooking convenience. Its four distinct cooking methods - steaming, baking, grilling and microwaving ensure your meals are cooked or reheated to perfection. Its multi-function capabilities can be combined to save time without compromising taste, texture or nutritional value. It’s the all-in-one kitchen companion designed for people with a busy lifestyle. The NN-CS89L offers next-level cooking convenience.
    1 These slim and stylish bodies are packed with high performance. The attractive compact designs and energy-saving functions help Panasonic Blu-ray products consume as little power as possible. You can experience great movie quality with this ultra-fast booting DMP-BD89 Full HD Blu-ray disc player. After starting the player, the time it takes from launching the menu to playing a disc is much shorter than in conventional models. The BD89 also allows for smart home networking (DLNA) and provides access to video on demand, so that home entertainment is more intuitive, more comfortable, and lots more fun. Panasonic DMP-BD89 Full HD Blu-ray disc player.