đ mt5-small-finetuned-28jan-2
This model is a fine - tuned version of google/mt5-small on an unknown dataset. It is designed for text summarization and can achieve certain performance on the evaluation set.
đ Quick Start
This model is a fine - tuned version of google/mt5-small on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.5078
- Rouge1: 18.7485
- Rouge2: 5.8034
- Rougel: 18.5163
- Rougelsum: 18.4817
đ Documentation
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
đ§ Technical Details
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 9
- eval_batch_size: 9
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
- lr_scheduler_type: linear
- num_epochs: 15
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Rouge1 |
Rouge2 |
Rougel |
Rougelsum |
| 5.9652 |
1.0 |
242 |
2.8048 |
14.036 |
4.037 |
13.7766 |
13.8254 |
| 3.474 |
2.0 |
484 |
2.7485 |
16.821 |
4.8051 |
16.6168 |
16.5782 |
| 3.2101 |
3.0 |
726 |
2.6444 |
17.2659 |
5.1077 |
16.9501 |
16.8998 |
| 3.0555 |
4.0 |
968 |
2.6408 |
17.3002 |
4.8657 |
17.0414 |
16.9794 |
| 2.9515 |
5.0 |
1210 |
2.5860 |
17.6468 |
5.3816 |
17.3755 |
17.3434 |
| 2.8694 |
6.0 |
1452 |
2.5586 |
18.3932 |
5.3896 |
18.2521 |
18.0748 |
| 2.7898 |
7.0 |
1694 |
2.5325 |
18.4954 |
5.5609 |
18.2994 |
18.2112 |
| 2.7436 |
8.0 |
1936 |
2.5431 |
18.8172 |
5.9338 |
18.4693 |
18.4324 |
| 2.6955 |
9.0 |
2178 |
2.5588 |
18.7895 |
6.1003 |
18.3593 |
18.3268 |
| 2.6571 |
10.0 |
2420 |
2.5079 |
19.2525 |
5.8268 |
19.0279 |
18.9846 |
| 2.629 |
11.0 |
2662 |
2.5118 |
18.9191 |
5.9877 |
18.6505 |
18.6 |
| 2.5998 |
12.0 |
2904 |
2.5070 |
18.7181 |
5.9061 |
18.4432 |
18.3931 |
| 2.5692 |
13.0 |
3146 |
2.5014 |
18.4412 |
6.1983 |
18.2394 |
18.1618 |
| 2.5751 |
14.0 |
3388 |
2.5125 |
18.7014 |
5.9729 |
18.4366 |
18.406 |
| 2.55 |
15.0 |
3630 |
2.5078 |
18.7485 |
5.8034 |
18.5163 |
18.4817 |
Framework versions
- Transformers 4.26.0
- Pytorch 1.13.1+cu116
- Datasets 2.9.0
- Tokenizers 0.13.2
đ License
This model is licensed under the Apache 2.0 license.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
