Wav2vec2 Xlsr 1b Finnish V2

🚀 Wav2Vec2 XLS-R for Finnish ASR

This acoustic model is designed for Finnish Automatic Speech Recognition (ASR). It's a fine - tuned version of facebook/wav2vec2-xls-r-1b, trained with 275.6 hours of Finnish transcribed speech data. Wav2Vec2 XLS - R was introduced in this paper and first released at this page.

Note: There's a version using the KenLM language model in the decoding phase, which produces better transcriptions: Finnish - NLP/wav2vec2-xlsr-1b-finnish-lm-v2

✨ Features

• Multilingual Pretraining: Wav2Vec2 XLS - R is a large - scale multilingual pretrained model for speech, pretrained on 436k hours of unlabeled speech across 128 languages, including VoxPopuli, MLS, CommonVoice, BABEL, and VoxLingua107.
• Fine - Tuned for Finnish: Specifically fine - tuned for Finnish ASR using a diverse set of Finnish speech datasets.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

No code examples are provided in the original document, so this section is skipped.

📚 Documentation

Model description

Wav2Vec2 XLS - R is Facebook AI's large - scale multilingual pretrained model for speech. It uses the wav2vec 2.0 objective and is pretrained on a vast amount of unlabeled speech data from multiple sources. This particular model is a fine - tuned variant of the 1 - billion - parameter version for Finnish ASR.

You can read more about the pretrained model from this blog and this paper.

Intended uses & limitations

How to use

Check the run - finnish - asr - models.ipynb notebook in this repository for a detailed example on how to use this model.

Limitations and bias

• Audio Length: The model was fine - tuned with audio samples of maximum 20 seconds. It likely works best for short audios of similar length. For very long audio files, you can use the audio chunking method introduced in [this blog post](https://huggingface.co/blog/asr - chunking) if you encounter out - of - memory errors.
• Domain Generalization: A large portion of the fine - tuning data was from the Finnish Parliament dataset. So, the model may not generalize well to different domains such as common daily spoken Finnish with dialects.
• Gender and Age Bias: The datasets' audios are mostly from adult males. Thus, the model may not perform as well for children and women's speeches.

Training data

This model was fine - tuned with 275.6 hours of Finnish transcribed speech data from the following datasets:

The datasets were filtered to include audio samples with a maximum length of 20 seconds.

Training procedure

This model was trained during the [Robust Speech Challenge Event](https://discuss.huggingface.co/t/open - to - the - community - robust - speech - recognition - challenge/13614) organized by Hugging Face. Training was done on a Tesla V100 GPU, sponsored by OVHcloud.

The training script was provided by Hugging Face and is available [here](https://github.com/huggingface/transformers/blob/main/examples/research_projects/robust - speech - event/run_speech_recognition_ctc_bnb.py). Only the data loading was modified for custom datasets.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: 8 - bit Adam with betas=(0.9, 0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10
• mixed_precision_training: Native AMP

The pretrained facebook/wav2vec2 - xls - r - 1b model was initialized with the following hyperparameters:

• attention_dropout: 0.094
• hidden_dropout: 0.047
• feat_proj_dropout: 0.04
• mask_time_prob: 0.082
• layerdrop: 0.041
• activation_dropout: 0.055
• ctc_loss_reduction: "mean"

Training results

Framework versions

• Transformers 4.17.0.dev0
• Pytorch 1.10.2+cu102
• Datasets 1.18.3
• Tokenizers 0.11.0

Evaluation results

Evaluation was done with the [Common Voice 7.0 Finnish test split](https://huggingface.co/datasets/mozilla - foundation/common_voice_7_0).

To evaluate this model, run the eval.py script in this repository:

python3 eval.py --model_id aapot/wav2vec2-xlsr-1b-finnish-v2 --dataset mozilla-foundation/common_voice_7_0 --config fi --split test

This model (the first row of the table) achieves the following WER (Word Error Rate) and CER (Character Error Rate) results compared to other models:

Team Members

• Aapo Tanskanen, Hugging Face profile, LinkedIn profile
• Rasmus Toivanen, Hugging Face profile, LinkedIn profile

Feel free to contact us for more details 🤗

🔧 Technical Details

The model is based on the Wav2Vec2 XLS - R architecture, which is a multilingual pretrained model for speech. The fine - tuning process involves adjusting the model's parameters to better fit the Finnish ASR task using a combination of hyperparameters and specific datasets. The use of 8 - bit Adam optimizer and Native AMP for mixed - precision training helps in efficient training on a Tesla V100 GPU.

📄 License

The model is licensed under the Apache 2.0 license.