Mt5 Small German Finetune Mlsum

🚀 mT5-small fine-tuned on German MLSUM

This model is fine-tuned on German MLSUM for text summarization, enhancing performance in generating concise summaries.

🚀 Quick Start

This model was finetuned for 3 epochs with a max_len (input) of 768 tokens and target_max_len of 192 tokens. It was fine-tuned on all German articles present in the train split of the MLSUM dataset having less than 384 "words" after splitting on whitespace, which resulted in 80249 articles.

The exact expression to filter the dataset was the following:

dataset = dataset.filter(lambda e: len(e['text'].split()) < 384)

📚 Documentation

Evaluation results

The fine-tuned model was evaluated on 2000 random articles from the validation set. Mean f1 ROUGE scores were calculated for both the fine-tuned model and the lead-3 baseline (which simply produces the leading three sentences of the document) and are presented in the following table.