đ mt5-small-finetuned-oneindia
This model is a fine - tuned version of google/mt5-small, which can generate summaries for news articles.
đ Quick Start
This model is a fine - tuned version of google/mt5-small on https://www.kaggle.com/datasets/akhisreelibra/malayalam-news dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0277
- Rouge1: 6.1146
- Rouge2: 0.9858
- Rougel: 6.085
- Rougelsum: 6.0965
⨠Features
- Model outputs summarised one liner to an article.
- Intended use in news domain, can be used to generate headlines to news articles.
đĻ Installation
No installation steps provided in the original document, so this section is skipped.
đģ Usage Examples
No code examples provided in the original document, so this section is skipped.
đ Documentation
Training and evaluation data
https://www.kaggle.com/datasets/akhisreelibra/malayalam-news
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5.6e - 05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
- lr_scheduler_type: linear
- num_epochs: 8
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Rouge1 |
Rouge2 |
Rougel |
Rougelsum |
| 3.1789 |
1.0 |
8994 |
2.2686 |
5.5492 |
0.8569 |
5.5448 |
5.5362 |
| 2.5859 |
2.0 |
17988 |
2.1751 |
5.7343 |
0.9775 |
5.7166 |
5.7195 |
| 2.4438 |
3.0 |
26982 |
2.1082 |
5.9718 |
0.9917 |
5.9503 |
5.9586 |
| 2.3584 |
4.0 |
35976 |
2.0730 |
6.1837 |
1.0286 |
6.1562 |
6.167 |
| 2.3015 |
5.0 |
44970 |
2.0512 |
6.0863 |
0.9656 |
6.0598 |
6.0643 |
| 2.2609 |
6.0 |
53964 |
2.0440 |
6.2119 |
0.9653 |
6.1859 |
6.1899 |
| 2.2344 |
7.0 |
62958 |
2.0303 |
6.2424 |
0.9835 |
6.2121 |
6.219 |
| 2.2166 |
8.0 |
71952 |
2.0277 |
6.1146 |
0.9858 |
6.085 |
6.0965 |
Framework versions
- Transformers 4.20.0
- Pytorch 1.11.0
- Datasets 2.1.0
- Tokenizers 0.12.1
đ§ Technical Details
No specific technical implementation details (more than 50 words) provided in the original document, so this section is skipped.
đ License
This model is licensed under the Apache 2.0 license.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
