Mt5 Multilingual XLSum Sumarizacao PTBR

🚀 mT5_multilingual_XLSum-sumarizacao-PTBR

This model is a fine - tuned variant of csebuetnlp/mT5_multilingual_XLSum on an unknown dataset. It offers valuable capabilities for text summarization tasks, achieving notable results on the evaluation set.

🚀 Quick Start

This model can be easily integrated into your projects. Here's how you can use it:

Basic Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

WHITESPACE_HANDLER = lambda k: re.sub('\s+', ' ', re.sub('\n+', ' ', k.strip()))

model_name = "GiordanoB/mT5_multilingual_XLSum-sumarizacao-PTBR"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

    input_ids = tokenizer(
        [WHITESPACE_HANDLER(sumariosDuplos[i])],
        return_tensors="pt",
        padding="max_length",
        truncation=True,
        max_length=512
    )["input_ids"]

    output_ids = model.generate(
        input_ids=input_ids,
        max_length=200,
        min_length=75,
        no_repeat_ngram_size=2,
        num_beams=5
    )[0]

    summary = tokenizer.decode(
        output_ids,
        skip_special_tokens=True,
        clean_up_tokenization_spaces=False
    )

    sumariosFinal.append(summary)
    print(i,"\n",summary,"\n")

✨ Features

This fine - tuned model achieves the following results on the evaluation set:

• Loss: 1.3870
• Rouge1: 42.0195
• Rouge2: 24.9493
• Rougel: 32.3653
• Rougelsum: 37.9982
• Gen Len: 77.0

📚 Documentation

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e - 05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Framework versions

• Transformers 4.18.0
• Pytorch 1.11.0
• Datasets 2.1.0
• Tokenizers 0.12.1