đ wav2vec2-xls-r-300m-kk-n2
This model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - KK dataset. It is designed for automatic speech recognition tasks, providing a reliable solution for transcribing Kazakh speech.
đ Quick Start
This model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - KK dataset. It achieves the following results on the evaluation set:
⨠Features
- Fine - Tuned: Based on the pre - trained facebook/wav2vec2-xls-r-300m, fine - tuned on the Kazakh dataset for better performance.
- Multiple Evaluation Metrics: Evaluated using Loss, WER (Word Error Rate), and CER (Character Error Rate).
đĻ Installation
No specific installation steps are provided in the original README.
đģ Usage Examples
Evaluation Commands
Evaluate on mozilla - foundation/common_voice_8_0 with test split
python eval.py --model_id DrishtiSharma/wav2vec2-xls-r-300m-kk-n2 --dataset mozilla-foundation/common_voice_8_0 --config kk --split test --log_outputs
Evaluate on speech - recognition - community - v2/dev_data
Kazakh language not found in speech-recognition-community-v2/dev_data!
đ Documentation
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.000222
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 150.0
- mixed_precision_training: Native AMP
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Wer |
| 9.6799 |
9.09 |
200 |
3.6119 |
1.0 |
| 3.1332 |
18.18 |
400 |
2.5352 |
1.005 |
| 1.0465 |
27.27 |
600 |
0.6169 |
0.682 |
| 0.3452 |
36.36 |
800 |
0.6572 |
0.607 |
| 0.2575 |
45.44 |
1000 |
0.6527 |
0.578 |
| 0.2088 |
54.53 |
1200 |
0.6828 |
0.551 |
| 0.158 |
63.62 |
1400 |
0.7074 |
0.5575 |
| 0.1309 |
72.71 |
1600 |
0.6523 |
0.5595 |
| 0.1074 |
81.8 |
1800 |
0.7262 |
0.5415 |
| 0.087 |
90.89 |
2000 |
0.7199 |
0.521 |
| 0.0711 |
99.98 |
2200 |
0.7113 |
0.523 |
| 0.0601 |
109.09 |
2400 |
0.6863 |
0.496 |
| 0.0451 |
118.18 |
2600 |
0.6998 |
0.483 |
| 0.0378 |
127.27 |
2800 |
0.6971 |
0.4615 |
| 0.0319 |
136.36 |
3000 |
0.7119 |
0.4475 |
| 0.0305 |
145.44 |
3200 |
0.7181 |
0.459 |
Framework versions
- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.2.dev0
- Tokenizers 0.11.0
đ§ Technical Details
The model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - KK dataset. It uses specific hyperparameters during training to optimize its performance on Kazakh speech recognition tasks.
đ License
The model is released under the Apache - 2.0 license.
| Property |
Details |
| Model Type |
Fine - tuned wav2vec2 - xls - r - 300m for Kazakh speech recognition |
| Training Data |
MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - KK dataset |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
