đ wav2vec2-large-xls-r-300m-hsb-v2
This model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - HSB dataset. It is designed for automatic speech recognition tasks, aiming to accurately transcribe speech in the Upper Sorbian (hsb) language.
⨠Features
- Multilingual Adaptation: Based on the large - scale wav2vec2 - xls - r - 300m model, it can be well - adapted to the HSB language.
- High - quality Performance: Achieves relatively low WER and CER on the evaluation set, indicating high recognition accuracy.
đĻ Installation
No specific installation steps are provided in the original document.
đģ Usage Examples
No code examples are provided in the original document.
đ Documentation
Evaluation Results
This model achieves the following results on the evaluation set:
Evaluation Commands
- To evaluate on mozilla - foundation/common_voice_8_0 with test split
python eval.py --model_id DrishtiSharma/wav2vec2-large-xls-r-300m-hsb-v2 --dataset mozilla-foundation/common_voice_8_0 --config hsb --split test --log_outputs
- To evaluate on speech - recognition - community - v2/dev_data
Upper Sorbian (hsb) not found in speech - recognition - community - v2/dev_data
Training Hyperparameters
The following hyperparameters were used during training:
| Property |
Details |
| Learning Rate |
0.00045 |
| Train Batch Size |
16 |
| Eval Batch Size |
8 |
| Seed |
42 |
| Gradient Accumulation Steps |
2 |
| Total Train Batch Size |
32 |
| Optimizer |
Adam with betas=(0.9,0.999) and epsilon = 1e - 08 |
| LR Scheduler Type |
linear |
| LR Scheduler Warmup Steps |
500 |
| Num Epochs |
50 |
| Mixed Precision Training |
Native AMP |
Training Results
| Training Loss |
Epoch |
Step |
Validation Loss |
Wer |
| 8.5979 |
3.23 |
100 |
3.5602 |
1.0 |
| 3.303 |
6.45 |
200 |
3.2238 |
1.0 |
| 3.2034 |
9.68 |
300 |
3.2002 |
0.9888 |
| 2.7986 |
12.9 |
400 |
1.2408 |
0.9210 |
| 1.3869 |
16.13 |
500 |
0.7973 |
0.7462 |
| 1.0228 |
19.35 |
600 |
0.6722 |
0.6788 |
| 0.8311 |
22.58 |
700 |
0.6100 |
0.6150 |
| 0.717 |
25.81 |
800 |
0.6236 |
0.6013 |
| 0.6264 |
29.03 |
900 |
0.6031 |
0.5575 |
| 0.5494 |
32.26 |
1000 |
0.5656 |
0.5309 |
| 0.4781 |
35.48 |
1100 |
0.5289 |
0.4996 |
| 0.4311 |
38.71 |
1200 |
0.5375 |
0.4768 |
| 0.3902 |
41.94 |
1300 |
0.5246 |
0.4703 |
| 0.3508 |
45.16 |
1400 |
0.5382 |
0.4696 |
| 0.3199 |
48.39 |
1500 |
0.5328 |
0.4596 |
Framework Versions
- Transformers 4.16.1
- Pytorch 1.10.0+cu111
- Datasets 1.18.2
- Tokenizers 0.11.0
đ§ Technical Details
The model is fine - tuned from the pre - trained wav2vec2 - xls - r - 300m model on the MOZILLA - FOUNDATION/COMMON_VOICE_8_0 - HSB dataset. Through specific hyperparameter settings and training processes, it can adapt to the characteristics of the HSB language and achieve good speech recognition performance.
đ License
This model is released under the Apache 2.0 license.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
