Wav2vec2 Xlsr 1b Finnish

🚀 Wav2Vec2 XLS-R for Finnish ASR

This acoustic model is a fine - tuned version of facebook/wav2vec2-xls-r-1b for Finnish Automatic Speech Recognition (ASR). It has been fine - tuned using 259.57 hours of Finnish transcribed speech data. Wav2Vec2 XLS - R was introduced in this paper and first released at this page.

Note: There is a version with a KenLM language model used in the decoding phase, which produces better transcriptions: Finnish - NLP/wav2vec2-xlsr-1b-finnish-lm.

Note: There is a better V2 version of this model, which has been fine - tuned for a longer time with 16 more hours of data: Finnish - NLP/wav2vec2-xlsr-1b-finnish-lm-v2.

✨ Features

• Fine - tuned from facebook/wav2vec2-xls-r-1b for Finnish ASR.
• Trained with 259.57 hours of Finnish transcribed speech data.
• There are alternative versions with language models and a more fine - tuned V2 version.

📚 Documentation

Model description

Wav2Vec2 XLS - R is a large - scale multilingual pretrained model for speech developed by Facebook AI. It is pretrained on 436k hours of unlabeled speech, including data from VoxPopuli, MLS, CommonVoice, BABEL, and VoxLingua107. It uses the wav2vec 2.0 objective and supports 128 languages.

You can read more about the pretrained model from this blog and this paper.

This model is a fine - tuned version of the pretrained model (1 billion parameter variant) for Finnish ASR.

Intended uses & limitations

How to use

Check the run - finnish - asr - models.ipynb notebook in this repository for a detailed example of how to use this model.

Limitations and bias

This model was fine - tuned with audio samples with a maximum length of 20 seconds. So, it most likely works best for relatively short audios of similar length. However, you can also try it with much longer audios and see how it performs. If you encounter out - of - memory errors with very long audio files, you can use the audio chunking method introduced in [this blog post](https://huggingface.co/blog/asr - chunking).

A vast majority of the data used for fine - tuning was from the Finnish Parliament dataset. So, this model may not generalize well to very different domains, such as common daily spoken Finnish with dialects. In addition, the datasets' audios tend to be dominated by adult males. So, this model may not work as well for the speeches of children and women, for example.

Training data

This model was fine - tuned with 259.57 hours of Finnish transcribed speech data from the following datasets:

The datasets were filtered to include audio samples with a maximum length of 20 seconds.

Training procedure

This model was trained during the [Robust Speech Challenge Event](https://discuss.huggingface.co/t/open - to - the - community - robust - speech - recognition - challenge/13614) organized by Hugging Face. The training was conducted on a Tesla V100 GPU, sponsored by OVHcloud.

The training script was provided by Hugging Face and is available [here](https://github.com/huggingface/transformers/blob/main/examples/research_projects/robust - speech - event/run_speech_recognition_ctc_bnb.py). We only modified its data loading for our custom datasets.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: 8 - bit Adam with betas=(0.9, 0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5
• mixed_precision_training: Native AMP

The pretrained facebook/wav2vec2-xls-r-1b model was initialized with the following hyperparameters:

• attention_dropout: 0.094
• hidden_dropout: 0.047
• feat_proj_dropout: 0.04
• mask_time_prob: 0.082
• layerdrop: 0.041
• activation_dropout: 0.055
• ctc_loss_reduction: "mean"

Training results

Framework versions

• Transformers 4.17.0.dev0
• Pytorch 1.10.2+cu102
• Datasets 1.18.3
• Tokenizers 0.11.0

Evaluation results

Evaluation was conducted using the [Common Voice 7.0 Finnish test split](https://huggingface.co/datasets/mozilla - foundation/common_voice_7_0).

To evaluate this model, run the eval.py script in this repository:

python3 eval.py --model_id aapot/wav2vec2-xlsr-1b-finnish --dataset mozilla-foundation/common_voice_7_0 --config fi --split test

This model (the second row of the table) achieves the following WER (Word Error Rate) and CER (Character Error Rate) results compared to our other models:

👥 Team Members

• Aapo Tanskanen, Hugging Face profile, LinkedIn profile
• Rasmus Toivanen, Hugging Face profile, LinkedIn profile

Feel free to contact us for more details 🤗

📄 License

This model is licensed under the Apache - 2.0 license.