TGL 3

🚀 TGL-3

TGL-3 is a fine - tuned model based on [t5 - small](https://huggingface.co/t5 - small), designed for text summarization tasks, trained on a dataset of 23000 abstract - summary data pieces from openreview.net.

🚀 Quick Start

This model is a fine - tuned version of [t5 - small](https://huggingface.co/t5 - small) on an abstract - summary dataset, with 23000 pieces of data for training. The data was acquired by openreview.net. It achieves the following results on the evaluation set:

• Loss: 2.4435
• Rouge1: 36.4998
• Rouge2: 17.8322
• Rougel: 31.8632
• Rougelsum: 31.8341

✨ Features

• Based on the pre - trained model [t5 - small](https://huggingface.co/t5 - small), it has strong generalization ability.
• Trained on a specific abstract - summary dataset, which is suitable for text summarization tasks.

📚 Documentation

Model description

Here is the paper https://arxiv.org/abs/1910.10683

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e - 05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Framework versions

• Transformers 4.21.1
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1

📄 License

This model is licensed under the Apache 2.0 license.