Flan T5 Base Summarization Pt Br

🚀 PamelaBorelli/flan-t5-base-summarization-pt-br

A Portuguese-language summarization model based on the Flan-T5 architecture, fine-tuned for text summarization tasks.

🚀 Quick Start

Basic Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("PamelaBorelli/flan-t5-base-summarization-pt-br")
model = T5ForConditionalGeneration.from_pretrained("PamelaBorelli/flan-t5-base-summarization-pt-br")

input_text = "O corpo está mais propenso a sentir dores com exercícios de alta intensidade | Foto: Getty Images O problema está em saber identificar qual é qual. \"Em algumas situações, é difícil diferenciar uma da outra\", reconhece Juan Francisco Marco, professor do Centro de Ciência do Esporte, Treinamento e Fitness Alto Rendimento, na Espanha. \"A dor boa é aquela que associamos ao exercício físico, que não limita (o movimento) e permite continuar (a se exercitar) até o momento em que o músculo fica realmente esgotado e não trabalha mais\", explica. É importante detectar qual é o tipo de dor que você está sentindo, para evitar ter problemas mais sérios | Foto: Getty Images Para Francisco Sánchez Diego, diretor do centro de treinamento Corpore 10, \"a dor boa se sente no grupo muscular que você trabalhou, tanto durante o treinamento como nos dias seguintes\"."
input_ids = tokenizer(input_text, return_tensors="pt").input_ids

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

✨ Features

• Multilingual Base: Built on the multilingual flan-t5-base with 248M parameters and a T5-based encoder-decoder architecture.
• Fine-tuned for Portuguese: Specifically fine-tuned for text summarization in Brazilian Portuguese through two finetuning steps.

📦 Installation

No specific installation steps are provided in the original document.

📚 Documentation

General Information

Summary

The original model that served as the basis for the final model is flan-t5-base. It is a multilingual model with 248M parameters and an architecture based on T5 (Text-to-Text Transfer Transformer) with an encoder-decoder. The original Flan-T5 was fine-tuned on a mixture of tasks to improve its generalization ability.

The final model PamelaBorelli/flan-t5-base-summarization-pt-br was trained through instruction and fine-tuning. Two fine-tuning steps were performed: first for text translation using a dataset and then for the summarization task with datasets in Brazilian Portuguese.

Intended Use

The model is designed for text summarization in Brazilian Portuguese and has not been tested for other languages.

Languages

Brazilian Portuguese

Training Data

The model was trained for the summarization task using the following parameters:

evaluation_strategy="steps"         # Way to evaluate the output
eval_steps=                         # Number of steps to evaluate the output
learning_rate=                      # Learning rate
per_device_train_batch_size=        # Batch size for training
per_device_eval_batch_size=         # Batch size for validation
gradient_accumulation_steps=        # How many batches to accumulate
weight_decay=                       # L2 regularization
num_train_epochs=                   # Number of training epochs
save_strategy="steps"               # Way to save the output
save_steps =                        # Number of steps to save the output
push_to_hub=False                   # Save the model on the Hugging Face hub
load_best_model_at_end=True         # Load the best model at the end of training (required for the callback)

For tokenization, the following parameters were used:

start_prompt= "Sumarize: \n"                 # Start instruction for summarization
end_prompt= "\n\nSumário: "    			 # End instruction for summarization
input_name="coluna_imput"           		 # Name of the column in the dataset with the original texts
target_name="coluna_target"          		 # Name of the column in the dataset with the target texts
max_input_length = 256         				 # Maximum input length for tokenization
max_target_length = 256        				 # Maximum target length for tokenization
columns_to_remove= ['coluna_to_remove'] 	 # Columns to be removed from the original dataset

📄 License

This model is licensed under the MIT license.