đ w2v-bert-2.0-naijavoices-clearglobal-hausa-500hr-v0
This model is a fine - tuned version of the transformers
library. It is a fine - tuned variant of facebook/w2v-bert-2.0 on the None dataset, offering enhanced performance in relevant tasks.
đ Quick Start
This section will guide you through the basic information and performance metrics of the w2v-bert-2.0-naijavoices-clearglobal-hausa-500hr-v0
model.
Model Information
Property |
Details |
Library Name |
transformers |
License |
MIT |
Base Model |
facebook/w2v-bert-2.0 |
Tags |
generated_from_trainer |
Metrics |
wer |
Evaluation Results
This model achieves the following results on the evaluation set:
- Loss: 0.1944
- Wer: 0.0747
- Cer: 0.0186
đ Documentation
Model description
This model is a fine - tuned version of facebook/w2v-bert-2.0 on the None dataset.
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
đ§ Technical Details
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 9e - 05
- train_batch_size: 64
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 128
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon = 1e - 08 and optimizer_args = No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.025
- num_epochs: 100.0
- mixed_precision_training: Native AMP
Training results
Training Loss |
Epoch |
Step |
Validation Loss |
Wer |
Cer |
0.389 |
1.0 |
2396 |
0.1232 |
0.1422 |
0.0334 |
0.1003 |
2.0 |
4792 |
0.1169 |
0.1378 |
0.0314 |
0.0975 |
3.0 |
7188 |
0.1177 |
0.1355 |
0.0319 |
0.0881 |
4.0 |
9584 |
0.1106 |
0.1231 |
0.0294 |
0.0831 |
5.0 |
11980 |
0.1112 |
0.1246 |
0.0302 |
0.079 |
6.0 |
14376 |
0.1123 |
0.1227 |
0.0297 |
0.0756 |
7.0 |
16772 |
0.1014 |
0.1160 |
0.0273 |
0.0735 |
8.0 |
19168 |
0.1042 |
0.1146 |
0.0273 |
0.0703 |
9.0 |
21564 |
0.0968 |
0.1096 |
0.0261 |
0.0667 |
10.0 |
23960 |
0.0967 |
0.1106 |
0.0260 |
0.0652 |
11.0 |
26356 |
0.1012 |
0.1121 |
0.0266 |
0.0613 |
12.0 |
28752 |
0.1010 |
0.1111 |
0.0266 |
0.0603 |
13.0 |
31148 |
0.1001 |
0.1109 |
0.0262 |
0.0575 |
14.0 |
33544 |
0.0937 |
0.1066 |
0.0249 |
0.0539 |
15.0 |
35940 |
0.0957 |
0.1079 |
0.0254 |
0.0542 |
16.0 |
38336 |
0.0993 |
0.1048 |
0.0250 |
0.0494 |
17.0 |
40732 |
0.0956 |
0.1023 |
0.0244 |
0.0471 |
18.0 |
43128 |
0.0995 |
0.1034 |
0.0243 |
0.0434 |
19.0 |
45524 |
0.0950 |
0.1025 |
0.0241 |
0.0411 |
20.0 |
47920 |
0.0992 |
0.1006 |
0.0239 |
0.0376 |
21.0 |
50316 |
0.1023 |
0.1017 |
0.0242 |
0.0339 |
22.0 |
52712 |
0.1015 |
0.0991 |
0.0234 |
0.0315 |
23.0 |
55108 |
0.1108 |
0.1005 |
0.0243 |
0.0281 |
24.0 |
57504 |
0.1100 |
0.0970 |
0.0230 |
0.0243 |
25.0 |
59900 |
0.1088 |
0.0937 |
0.0224 |
0.0214 |
26.0 |
62296 |
0.1162 |
0.0929 |
0.0222 |
0.0196 |
27.0 |
64692 |
0.1173 |
0.0920 |
0.0221 |
0.0173 |
28.0 |
67088 |
0.1262 |
0.0928 |
0.0225 |
0.0145 |
29.0 |
69484 |
0.1252 |
0.0913 |
0.0222 |
0.0132 |
30.0 |
71880 |
0.1318 |
0.0877 |
0.0214 |
0.0115 |
31.0 |
74276 |
0.1342 |
0.0868 |
0.0211 |
0.0114 |
32.0 |
76672 |
0.1326 |
0.0884 |
0.0216 |
0.0098 |
33.0 |
79068 |
0.1352 |
0.0853 |
0.0210 |
0.0088 |
34.0 |
81464 |
0.1398 |
0.0856 |
0.0209 |
0.008 |
35.0 |
83860 |
0.1432 |
0.0860 |
0.0210 |
0.0072 |
36.0 |
86256 |
0.1475 |
0.0840 |
0.0207 |
0.0065 |
37.0 |
88652 |
0.1444 |
0.0836 |
0.0206 |
0.006 |
38.0 |
91048 |
0.1467 |
0.0845 |
0.0208 |
0.006 |
39.0 |
93444 |
0.1501 |
0.0854 |
0.0210 |
0.0055 |
40.0 |
95840 |
0.1545 |
0.0841 |
0.0209 |
0.0047 |
41.0 |
98236 |
0.1567 |
0.0820 |
0.0201 |
0.0048 |
42.0 |
100632 |
0.1507 |
0.0817 |
0.0200 |
0.0043 |
43.0 |
103028 |
0.1520 |
0.0817 |
0.0202 |
0.0043 |
44.0 |
105424 |
0.1522 |
0.0836 |
0.0206 |
0.0037 |
45.0 |
107820 |
0.1559 |
0.0801 |
0.0198 |
0.0036 |
46.0 |
110216 |
0.1588 |
0.0797 |
0.0197 |
0.0036 |
47.0 |
112612 |
0.1562 |
0.0788 |
0.0196 |
0.003 |
48.0 |
115008 |
0.1674 |
0.0791 |
0.0196 |
0.0031 |
49.0 |
117404 |
0.1682 |
0.0787 |
0.0197 |
0.0029 |
50.0 |
119800 |
0.1612 |
0.0787 |
0.0197 |
0.0029 |
51.0 |
122196 |
0.1548 |
0.0802 |
0.0201 |
0.0025 |
52.0 |
124592 |
0.1630 |
0.0771 |
0.0192 |
0.0023 |
53.0 |
126988 |
0.1654 |
0.0792 |
0.0198 |
0.0021 |
54.0 |
129384 |
0.1754 |
0.0780 |
0.0193 |
0.002 |
55.0 |
131780 |
0.1687 |
0.0777 |
0.0193 |
0.002 |
56.0 |
134176 |
0.1746 |
0.0764 |
0.0191 |
0.0018 |
57.0 |
136572 |
0.1655 |
0.0765 |
0.0192 |
0.0019 |
58.0 |
138968 |
0.1688 |
0.0794 |
0.0200 |
0.0017 |
59.0 |
141364 |
0.1681 |
0.0753 |
0.0189 |
0.0015 |
60.0 |
143760 |
0.1782 |
0.0767 |
0.0191 |
0.0014 |
61.0 |
146156 |
0.1772 |
0.0773 |
0.0194 |
0.0014 |
62.0 |
148552 |
0.1839 |
0.0752 |
0.0189 |
0.0014 |
63.0 |
150948 |
0.1781 |
0.0765 |
0.0190 |
0.0013 |
64.0 |
153344 |
0.1809 |
0.0764 |
0.0192 |
0.0012 |
65.0 |
155740 |
0.1793 |
0.0767 |
0.0192 |
0.0011 |
66.0 |
158136 |
0.1766 |
0.0766 |
0.0194 |
0.001 |
67.0 |
160532 |
0.1832 |
0.0747 |
0.0186 |
0.001 |
68.0 |
162928 |
0.1845 |
0.0762 |
0.0191 |
0.0009 |
69.0 |
165324 |
0.1944 |
0.0747 |
0.0186 |
Framework versions
- Transformers 4.50.3
- Pytorch 2.6.0+cu124
- Datasets 3.5.0
- Tokenizers 0.21.1
đ License
This project is licensed under the MIT license.