Mt5 Small Sum De En V2

🚀 mT5-small-sum-de-en-v2

This is a bilingual summarization model for English and German, built upon the multilingual T5 model google/mt5-small, aiming to provide efficient summarization solutions for both languages.

🚀 Quick Start

This bilingual summarization model offers seamless summarization capabilities for English and German. It's based on the google/mt5-small model, leveraging its multilingual features.

✨ Features

• Bilingual Summarization: Specifically designed for English and German, catering to a wide range of users.
• Multilingual Base: Built on the google/mt5-small model, benefiting from its pre - trained multilingual knowledge.

🔧 Technical Details

Training

The training was carried out with the following hyperparameters:

• Base Model: google/mt5-small
• Source Prefix: "summarize: "
• Batch Size: 3
• Max Source Length: 800
• Max Target Length: 96
• Warmup Ratio: 0.3
• Number of Train Epochs: 10
• Gradient Accumulation Steps: 2
• Learning Rate: 5e - 5

Datasets and Preprocessing

The datasets were preprocessed as follows: The summary was tokenized using the google/mt5-small tokenizer. Then, only the records with no more than 94 summary tokens were selected.

The MLSUM dataset has a special characteristic. In the text, the summary is often included completely as one or more sentences. These have been removed from the texts to avoid training a model that simply extracts sentences as summaries.

This model is trained on the following datasets:

Evaluation

Evaluation on MLSUM German Test Set (no beams)

Evaluation on CNN Daily English Test Set (no beams)

Evaluation on Extreme Summarization (XSum) English Test Set (no beams)

♣: These values seem to be unusually high. It could be that the test set was used in the training data.

📄 License

Copyright (c) 2021 Philip May, T-Systems on site services GmbH

This work is licensed under the [Attribution - NonCommercial - ShareAlike 3.0 Unported (CC BY - NC - SA 3.0)](https://creativecommons.org/licenses/by - nc - sa/3.0/) license.