mt5-small-sum-de-en-v1 Open-source Summary Model - Free Deployment for English-German Text Summarization

Mt5 Small Sum De En V1

Developed by deutsche-telekom

A bilingual summarization model based on multilingual T5, supporting English and German text summarization tasks

Supports Multiple Languages#German-English Bilingual Summarization #Multi-dataset Training #News Summarization Generation

Downloads 1,210

Release Time : 3/2/2022

Model Overview

This model was developed by Deutsche Telekom's One Conversation team, specializing in English and German text summarization tasks, and has demonstrated excellent performance on multiple public datasets

Model Features

Bilingual Support

Supports text summarization tasks in both English and German languages

Multi-dataset Training

Trained on 6 different datasets with over 900,000 training samples in total

Performance Optimization

Outperforms similar models on the MLSUM German test set and CNN English test set

Data Preprocessing

Special processing for the MLSUM dataset to prevent the model from learning to extract pre-existing sentences

Model Capabilities

English Text Summarization

German Text Summarization

Multilingual Text Processing

Use Cases

News Summarization

English News Summarization

Generates concise summaries for English news content such as CNN Daily Mail

Achieved Rouge-1 score of 37.63 on the CNN test set

German News Summarization

Generates summaries for German news content such as MLSUM

Achieved Rouge-1 score of 21.73 on the MLSUM test set

Wikipedia Summarization

Wikipedia Content Summarization

Generates summaries for English and German Wikipedia content

🚀 mT5-small-sum-de-en-v1

This is a bilingual summarization model for English and German, based on the multilingual T5 model google/mt5-small. It offers efficient and accurate summarization capabilities for both languages.

This model is provided by the One Conversation team of Deutsche Telekom AG.

🚀 Quick Start

This section provides a high - level overview of the model's capabilities and how it can be used for bilingual summarization.

✨ Features

Bilingual Summarization: Supports both English and German summarization tasks.
Based on mT5: Leverages the power of the multilingual T5 model google/mt5-small.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

📚 Documentation

🔧 Training

The training was conducted with the following hyperparameters:

Base Model: google/mt5-small
Source Prefix: "summarize: "
Batch Size: 3
Max Source Length: 800
Max Target Length: 96
Warmup Ratio: 0.3
Number of Train Epochs: 10
Gradient Accumulation Steps: 2
Learning Rate: 5e - 5

📊 Datasets and Preprocessing

The datasets were preprocessed as follows: The summary was tokenized with the google/mt5-small tokenizer. Then only the records with no more than 94 summary tokens were selected.

The MLSUM dataset has a special characteristic. In the text, the summary is often included completely as one or more sentences. These have been removed from the texts. The reason is that we do not want to train a model that ultimately extracts only sentences as a summary.

This model is trained on the following datasets:

Name	Language	Size	License
CNN Daily - Train	en	218,223	The license is unclear. The data comes from CNN and Daily Mail. We assume that it may only be used for research purposes and not commercially.
Extreme Summarization (XSum) - Train	en	204,005	The license is unclear. The data comes from BBC. We assume that it may only be used for research purposes and not commercially.
wiki_lingua English	en	130,331	Creative Commons CC BY - NC - SA 3.0 License
wiki_lingua German	de	48,390	Creative Commons CC BY - NC - SA 3.0 License
MLSUM German - Train	de	218,043	Usage of dataset is restricted to non - commercial research purposes only. Copyright belongs to the original copyright holders (see here).
SwissText 2019 - Train	de	84,564	The license is unclear. The data was published in the German Text Summarization Challenge. We assume that they may be used for research purposes and not commercially.

Language	Size
German	350,997
English	552,559
Total	903,556

📈 Evaluation

Evaluation on MLSUM German Test Set (no beams)

Model	rouge1	rouge2	rougeL	rougeLsum
ml6team/mt5-small-german-finetune-mlsum	18.3607	5.3604	14.5456	16.1946
deutsche - telekom/mT5 - small - sum - de - en - 01 (this)	21.7336	7.2614	17.1323	19.3977

Evaluation on CNN Daily English Test Set (no beams)

Model	rouge1	rouge2	rougeL	rougeLsum
sshleifer/distilbart-xsum-12-6	26.7664	8.8243	18.3703	23.2614
facebook/bart-large-xsum	28.5374	9.8565	19.4829	24.7364
mrm8488/t5-base-finetuned-summarize-news	37.576	14.7389	24.0254	34.4634
deutsche - telekom/mT5 - small - sum - de - en - 01 (this)	37.6339	16.5317	27.1418	34.9951

Evaluation on Extreme Summarization (XSum) English Test Set (no beams)

Model	rouge1	rouge2	rougeL	rougeLsum
mrm8488/t5-base-finetuned-summarize-news	18.6204	3.535	12.3997	15.2111
facebook/bart-large-xsum	28.5374	9.8565	19.4829	24.7364
deutsche - telekom/mT5 - small - sum - de - en - 01 (this)	32.3416	10.6191	25.3799	25.3908
sshleifer/distilbart-xsum-12-6	44.2553 ♣	21.4289 ♣	36.2639 ♣	36.2696 ♣

♣: These values seem to be unusually high. It could be that the test set was used in the training data.

📄 License

This work is licensed under the [Attribution - NonCommercial - ShareAlike 3.0 Unported (CC BY - NC - SA 3.0)](https://creativecommons.org/licenses/by - nc - sa/3.0/) license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご