Pegasus Summarization

🚀 Using HuggingFace Transformers for Summarization Task

This project demonstrates how to use HuggingFace Transformers for text summarization tasks. It provides code examples and decoder configuration examples to help you generate summaries effectively.

🚀 Quick Start

Prerequisites

Make sure you have installed the transformers library.

Installation

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("AlekseyKulnevich/Pegasus-Summarization")
# There is a mistake here, should be AutoTokenizer instead of PegasusTokenizer
tokenizer = AutoTokenizer.from_pretrained('google/pegasus-large')
input_text = # your text 
input_ = tokenizer.batch_encode_plus([input_text], max_length=1024, pad_to_max_length=True, 
                truncation=True, padding='longest', return_tensors='pt')
input_ids = input_['input_ids'] 
input_mask = input_['attention_mask']
summary = model.generate(input_ids=input_ids, 
                         attention_mask=input_mask, 
                         num_beams=32, 
                         min_length=100,
                         no_repeat_ngram_size=2, 
                         early_stopping=True, 
                         num_return_sequences=10)
questions = tokenizer.batch_decode(summary, skip_special_tokens=True)

Decoder Configuration Examples

You can find the input text here.

Example 1

summary = model.generate(input_ids=input_ids, 
                         attention_mask=input_mask, 
                         num_beams=32, 
                         min_length=100,
                         no_repeat_ngram_size=2, 
                         early_stopping=True, 
                         num_return_sequences=1)

tokenizer.batch_decode(summary, skip_special_tokens=True)

Output:

1. global warming will expand the range of tropical cyclones in the mid-latitudes of the world, according to a new study published by the Intergovernmental Panel on Climate change (IPCC) and the US National Oceanic and Atmospheric Administration (NOAA) The study shows that a warming climate will allow more of these types of storms to form over a wider range than they have been able to do over the past three million years. "As the climate warms, it's likely that these storms will become more frequent and more intense," said the authors of this study.

Example 2

summary = model.generate(input_ids=input_ids, 
                         attention_mask=input_mask, 
                         top_k=30, 
                         no_repeat_ngram_size=2, 
                         early_stopping=True, 
                         min_length=100,
                         num_return_sequences=1)

tokenizer.batch_decode(summary, skip_special_tokens=True)

Output:

1. tropical cyclones in the mid-latitudes of the world will likely form more of these types of storms, according to a new study published by the Intergovernmental Panel on Climate change (IPCC) on the impact of human induced climate change on these storms. The study shows that a warming climate will increase the likelihood of a subtropical cyclone forming over a wider range of latitudes, including the equator, than it has been for the past three million years, and that it will be more likely to form over the tropics.

Advanced Usage

You can also play with the following parameters in the generate method:

• top_k
• top_p

You can find the meaning of parameters to generate text here.