🚀 common8
This model is a fine - tuned version of wghts/checkpoint-20000 on the MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - FA dataset. It can achieve specific results on the evaluation set, providing more accurate speech recognition capabilities.
🚀 Quick Start
This model is a fine - tuned version of wghts/checkpoint-20000 on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - FA dataset.
It achieves the following results on the evaluation set:
✨ Features
Model Performance
It achieves a Loss of 0.3174 and a Wer of 0.3022 on the evaluation set, demonstrating good performance in automatic speech recognition.
Fine - Tuning
Based on the pre - trained model wghts/checkpoint-20000, fine - tuned on the MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - FA dataset to better adapt to specific language scenarios.
🔧 Technical Details
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e - 06
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 6
- total_train_batch_size: 192
- optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 250.0
- mixed_precision_training: Native AMP
Training results
Training Loss |
Epoch |
Step |
Validation Loss |
Wer |
3.5847 |
1.93 |
500 |
3.5104 |
1.0 |
2.7858 |
3.86 |
1000 |
2.9601 |
1.0001 |
1.6827 |
5.79 |
1500 |
0.7853 |
0.7030 |
1.4656 |
7.72 |
2000 |
0.6076 |
0.6014 |
1.3693 |
9.65 |
2500 |
0.5114 |
0.5307 |
1.379 |
11.58 |
3000 |
0.4666 |
0.4940 |
1.2832 |
13.51 |
3500 |
0.4257 |
0.4593 |
1.1931 |
15.44 |
4000 |
0.4039 |
0.4427 |
1.2911 |
17.37 |
4500 |
0.3956 |
0.4295 |
1.1577 |
19.3 |
5000 |
0.3705 |
0.4114 |
1.1135 |
21.24 |
5500 |
0.3740 |
0.4010 |
1.19 |
23.17 |
6000 |
0.3611 |
0.3935 |
1.1008 |
25.1 |
6500 |
0.3503 |
0.3880 |
1.0805 |
27.03 |
7000 |
0.3427 |
0.3781 |
1.1556 |
28.96 |
7500 |
0.3442 |
0.3727 |
1.0596 |
30.89 |
8000 |
0.3398 |
0.3646 |
1.0219 |
32.82 |
8500 |
0.3312 |
0.3660 |
1.1042 |
34.75 |
9000 |
0.3287 |
0.3612 |
1.0273 |
36.68 |
9500 |
0.3236 |
0.3556 |
1.0383 |
38.61 |
10000 |
0.3217 |
0.3558 |
1.0498 |
40.54 |
10500 |
0.3205 |
0.3520 |
0.9969 |
42.47 |
11000 |
0.3125 |
0.3504 |
1.0658 |
44.4 |
11500 |
0.3120 |
0.3493 |
0.992 |
46.33 |
12000 |
0.3137 |
0.3476 |
0.9737 |
48.26 |
12500 |
0.3085 |
0.3413 |
1.0817 |
50.19 |
13000 |
0.3091 |
0.3418 |
0.9414 |
52.12 |
13500 |
0.3072 |
0.3344 |
0.9295 |
54.05 |
14000 |
0.3039 |
0.3322 |
1.0248 |
55.98 |
14500 |
0.2991 |
0.3325 |
0.9474 |
57.91 |
15000 |
0.3032 |
0.3348 |
0.928 |
59.85 |
15500 |
0.2999 |
0.3285 |
1.0321 |
61.78 |
16000 |
0.2982 |
0.3253 |
0.9255 |
63.71 |
16500 |
0.2970 |
0.3231 |
0.8928 |
65.64 |
17000 |
0.2993 |
0.3250 |
1.008 |
67.57 |
17500 |
0.2985 |
0.3222 |
0.9371 |
69.5 |
18000 |
0.2968 |
0.3216 |
0.9077 |
71.43 |
18500 |
0.3011 |
0.3299 |
1.0044 |
73.36 |
19000 |
0.3053 |
0.3306 |
0.9625 |
75.29 |
19500 |
0.3159 |
0.3295 |
0.9816 |
77.22 |
20000 |
0.3080 |
0.3304 |
0.9587 |
119.19 |
20500 |
0.3088 |
0.3284 |
0.9178 |
122.09 |
21000 |
0.3132 |
0.3320 |
1.0282 |
125.0 |
21500 |
0.3099 |
0.3266 |
0.9337 |
127.9 |
22000 |
0.3110 |
0.3317 |
0.8822 |
130.81 |
22500 |
0.3037 |
0.3247 |
0.9644 |
133.72 |
23000 |
0.3037 |
0.3238 |
0.9214 |
136.62 |
23500 |
0.3040 |
0.3234 |
0.9167 |
139.53 |
24000 |
0.3079 |
0.3203 |
0.9047 |
142.44 |
24500 |
0.3018 |
0.3177 |
0.8909 |
145.35 |
25000 |
0.3053 |
0.3181 |
0.9646 |
148.25 |
25500 |
0.3095 |
0.3229 |
0.8802 |
151.16 |
26000 |
0.3111 |
0.3192 |
0.8411 |
154.07 |
26500 |
0.3068 |
0.3123 |
0.9235 |
156.97 |
27000 |
0.3090 |
0.3177 |
0.8943 |
159.88 |
27500 |
0.3115 |
0.3179 |
0.8854 |
162.79 |
28000 |
0.3052 |
0.3157 |
0.8734 |
165.69 |
28500 |
0.3077 |
0.3124 |
0.8515 |
168.6 |
29000 |
0.3117 |
0.3128 |
0.912 |
171.51 |
29500 |
0.3039 |
0.3121 |
0.8669 |
174.42 |
30000 |
0.3120 |
0.3123 |
0.823 |
177.32 |
30500 |
0.3148 |
0.3118 |
0.9129 |
180.23 |
31000 |
0.3179 |
0.3101 |
0.8255 |
183.14 |
31500 |
0.3164 |
0.3114 |
0.8948 |
186.05 |
32000 |
0.3128 |
0.3101 |
0.8397 |
188.95 |
32500 |
0.3143 |
0.3068 |
0.8341 |
191.86 |
33000 |
0.3127 |
0.3136 |
0.873 |
194.76 |
33500 |
0.3149 |
0.3124 |
0.8232 |
197.67 |
34000 |
0.3166 |
0.3086 |
0.8002 |
200.58 |
34500 |
0.3149 |
0.3061 |
0.8621 |
203.49 |
35000 |
0.3160 |
0.3093 |
0.8123 |
206.39 |
35500 |
0.3141 |
0.3063 |
0.7995 |
209.3 |
36000 |
0.3174 |
0.3075 |
0.8271 |
212.21 |
36500 |
0.3173 |
0.3043 |
0.8059 |
215.12 |
37000 |
0.3176 |
0.3079 |
0.8835 |
218.02 |
37500 |
0.3169 |
0.3062 |
0.8027 |
220.93 |
38000 |
0.3203 |
0.3098 |
0.775 |
223.83 |
38500 |
0.3159 |
0.3068 |
0.8487 |
226.74 |
39000 |
0.3161 |
0.3072 |
0.7929 |
229.65 |
39500 |
0.3143 |
0.3037 |
0.7653 |
232.56 |
40000 |
0.3160 |
0.3048 |
0.8211 |
235.46 |
40500 |
0.3173 |
0.3031 |
0.7761 |
238.37 |
41000 |
0.3176 |
0.3025 |
0.7761 |
241.28 |
41500 |
0.3179 |
0.3027 |
0.7903 |
244.19 |
42000 |
0.3181 |
0.3016 |
0.7807 |
247.09 |
42500 |
0.3170 |
0.3027 |
0.8406 |
250.0 |
43000 |
0.3174 |
0.3022 |
Framework versions
- Transformers 4.17.0.dev0
- Pytorch 1.10.2
- Datasets 1.18.3.dev0
- Tokenizers 0.10.3