T5 Base Dutch Demo

🚀 t5-base-dutch-demo 📰

This project was created by Yeb Havinga and Dat Nguyen during the Hugging Face community week. It is a model based on t5-base-dutch, fine - tuned to generate summaries of news articles. You can check out the model demo in the Netherformer 📰 example application on Hugging Face Spaces.

🚀 Quick Start

For a quick look at the model's capabilities, head over to the Hugging Face Spaces for the Netherformer 📰 example application.

✨ Features

• Based on the t5-base-dutch model.
• Fine - tuned to create summaries of news articles.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

No code examples are provided in the original document, so this section is skipped.

📚 Documentation

Dataset

t5-base-dutch-demo is fine - tuned on three mixed news sources:

1. CNN DailyMail: Translated to Dutch with MarianMT.
2. XSUM: Translated to Dutch with MarianMt.
3. News article summaries distilled from the nu.nl website.

The total number of training examples in this dataset is 1366592.

Training

Training involved fine - tuning t5-base-dutch with the following parameters:

• Constant learning rate: 0.0005
• Batch size: 8
• 1 epoch (170842 steps)

Evaluation

The performance of the summarization model is measured with the Rouge metric from the Huggingface Datasets library.

    "rouge{n}" (e.g. `"rouge1"`, `"rouge2"`) where: {n} is the n - gram based scoring,
    "rougeL": Longest common subsequence based scoring.

• Rouge1: 23.8
• Rouge2: 6.9
• RougeL: 19.7

These scores are expected to improve if the model is trained with evaluation configured for the CNN DM and XSUM datasets (translated to Dutch) individually.

🔧 Technical Details

The model is based on t5-base-dutch and fine - tuned on a specific dataset. The training process uses a constant learning rate, a specific batch size, and a single epoch with a certain number of steps. Evaluation is done using the Rouge metric from the Huggingface Datasets library.

📄 License

No license information is provided in the original document, so this section is skipped.