đ d-l-dl
This model is a fine - tuned version of facebook/wav2vec2-base-960h on an unknown dataset. It offers a practical solution for speech - related tasks by leveraging the pre - trained capabilities of the base model and fine - tuning them.
đ Quick Start
This model can be used in speech - related tasks. You can load it using relevant deep - learning frameworks and start inference or further fine - tuning.
⨠Features
- Based on the pre - trained [facebook/wav2vec2-base-960h] model, which has been proven effective in speech processing.
- Fine - tuned on an unknown dataset to potentially adapt to specific speech characteristics.
đĻ Installation
No specific installation steps are provided in the original document.
đģ Usage Examples
No code examples are provided in the original document.
đ Documentation
Model description
This model is a fine - tuned version of [facebook/wav2vec2-base-960h] on an unknown dataset. However, more detailed information about the model's architecture, design philosophy, and specific application scenarios needs to be further explored.
Intended uses & limitations
The original document does not provide specific information about the intended uses and limitations of this model.
Training and evaluation data
The original document does not provide details about the training and evaluation data, such as the source, size, and characteristics of the dataset.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 800
- mixed_precision_training: Native AMP
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Wer |
| 42.4143 |
49.8 |
100 |
21.5116 |
1.0 |
| 5.9884 |
99.8 |
200 |
31.7976 |
1.0 |
| 4.0043 |
149.8 |
300 |
3.4829 |
1.0 |
| 3.653 |
199.8 |
400 |
3.6417 |
1.0 |
| 3.5207 |
249.8 |
500 |
3.5081 |
1.0 |
| 3.63 |
299.8 |
600 |
3.4836 |
1.0 |
| 3.648 |
349.8 |
700 |
3.4515 |
1.0 |
| 3.6448 |
399.8 |
800 |
3.4647 |
1.0 |
| 3.6872 |
449.8 |
900 |
3.4371 |
1.0 |
| 3.6892 |
499.8 |
1000 |
3.4337 |
1.0 |
| 3.684 |
549.8 |
1100 |
3.4375 |
1.0 |
| 3.6843 |
599.8 |
1200 |
3.4452 |
1.0 |
| 3.6842 |
649.8 |
1300 |
3.4416 |
1.0 |
| 3.6819 |
699.8 |
1400 |
3.4498 |
1.0 |
| 3.6832 |
749.8 |
1500 |
3.4524 |
1.0 |
| 3.6828 |
799.8 |
1600 |
3.4495 |
1.0 |
Framework versions
- Transformers 4.11.3
- Pytorch 1.10.0+cu113
- Datasets 1.18.3
- Tokenizers 0.10.3
đ§ Technical Details
The original document does not provide in - depth technical details about the model's design, algorithm, and implementation.
đ License
The original document does not provide license information.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
