I am using huggingface transformer models for text-summarization. Currently I am testing different models such as T5 and Pegasus. Now these models were trained for summarizing Big Texts into very short like a maximum of two sentences. Now I have the task, that I want summarizations, that are about half the size of the text, ergo the generated summaries are too small for my purpose.
My question now is, if there is a way to tell the model that another sentence came before? Kind of similar to the logic inside stateful RNNs (although I know they work completly different). If yes, I could summarize small windows over the sentences always with the information which content came before.
Is that just a thing of my mind? I cant believe that I am the only one, who wants to create shorter summaries, but not only 1 or two sentence long ones.
Thank you
Why not transfer learning? Train them on your specific texts and summaries.
I trained T5 on specific limited text over 5 epoch and got very good results. I adopted the code from here to my needs https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb
Let me know if you have a specific training questions.