Wav2vec2 Xlsr 300m Finnish

🚀 Wav2Vec2 XLS-R for Finnish ASR

This acoustic model is a fine - tuned version of facebook/wav2vec2-xls-r-300m for Finnish Automatic Speech Recognition (ASR), providing high - quality speech - to - text conversion for the Finnish language.

✨ Features

• Fine - Tuned for Finnish: Based on the pre - trained facebook/wav2vec2-xls-r-300m, fine - tuned with 275.6 hours of Finnish transcribed speech data.
• Multilingual Pretrained Base: Wav2Vec2 XLS - R is pretrained on 436k hours of unlabeled speech in 128 languages, including VoxPopuli, MLS, CommonVoice, BABEL, and VoxLingua107.
• Available with LM Version: There is a version with KenLM language model used in the decoding phase, which can produce better transcriptions: [Finnish - NLP/wav2vec2-xlsr-300m-finnish-lm](https://huggingface.co/Finnish - NLP/wav2vec2-xlsr-300m-finnish-lm).

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

No code examples are provided in the original document, so this section is skipped.

📚 Documentation

Intended uses & limitations

• Intended Use: This model can be used for Finnish ASR (speech - to - text) task.
• How to Use: Check the [run - finnish - asr - models.ipynb](https://huggingface.co/aapot/wav2vec2-xlsr-300m-finnish/blob/main/run - finnish - asr - models.ipynb) notebook in this repository for a detailed example on how to use this model.
• Limitations and bias:
  • Audio Length: This model was fine - tuned with audio samples of maximum length 20 seconds, so it most likely works best for short audios of similar length. However, you can try it with longer audios. If you encounter out - of - memory errors with very long audio files, you can use the audio chunking method introduced in [this blog post](https://huggingface.co/blog/asr - chunking).
  • Data Domain: A vast majority of the data used for fine - tuning was from the Finnish Parliament dataset, so this model may not generalize well to very different domains like common daily spoken Finnish with dialects. Also, audios of the datasets tend to be adult male - dominated, so this model may not work as well for speeches of children and women.

Training data

This model was fine - tuned with 275.6 hours of Finnish transcribed speech data from the following datasets:

Datasets were filtered to include audio samples with a maximum length of 20 seconds.

Training procedure

This model was trained during the [Robust Speech Challenge Event](https://discuss.huggingface.co/t/open - to - the - community - robust - speech - recognition - challenge/13614) organized by Hugging Face. Training was done on a Tesla V100 GPU, sponsored by OVHcloud. The training script was provided by Hugging Face and is available [here](https://github.com/huggingface/transformers/blob/main/examples/research_projects/robust - speech - event/run_speech_recognition_ctc_bnb.py). Only the data loading was modified for custom datasets.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 04
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: 8 - bit Adam with betas=(0.9, 0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10
• mixed_precision_training: Native AMP

The pretrained facebook/wav2vec2 - xls - r - 300m model was initialized with the following hyperparameters:

• attention_dropout: 0.094
• hidden_dropout: 0.047
• feat_proj_dropout: 0.04
• mask_time_prob: 0.082
• layerdrop: 0.041
• activation_dropout: 0.055
• ctc_loss_reduction: "mean"

Training results

Framework versions

• Transformers 4.17.0.dev0
• Pytorch 1.10.2+cu102
• Datasets 1.18.3
• Tokenizers 0.11.0

Evaluation results

Evaluation was done with the [Common Voice 7.0 Finnish test split](https://huggingface.co/datasets/mozilla - foundation/common_voice_7_0). To evaluate this model, run the eval.py script in this repository:

python3 eval.py --model_id aapot/wav2vec2-xlsr-300m-finnish --dataset mozilla-foundation/common_voice_7_0 --config fi --split test

This model (the third row of the table) achieves the following WER (Word Error Rate) and CER (Character Error Rate) results compared to other models:

Team Members

• Aapo Tanskanen, Hugging Face profile, LinkedIn profile
• Rasmus Toivanen, Hugging Face profile, LinkedIn profile

Feel free to contact us for more details 🤗

📄 License

This project is licensed under the Apache - 2.0 license.