Wav2vec2 Xlsr 1b Finnish Lm

🚀 Wav2vec2-xls-r-1b for Finnish ASR

This acoustic model is a fine - tuned version of facebook/wav2vec2-xls-r-1b for Finnish Automatic Speech Recognition (ASR). It has been fine - tuned using 259.57 hours of Finnish transcribed speech data. Wav2Vec2 XLS - R was introduced in this paper and first released at this page.

This repository also includes a Finnish KenLM language model used in the decoding phase with the acoustic model.

Note: this model is identical to the aapot/wav2vec2-xlsr-1b-finnish-lm model; it has just been copied/moved to this Finnish - NLP Hugging Face organization.

Note: there is a better V2 version of this model, Finnish-NLP/wav2vec2-xlsr-1b-finnish-lm-v2, which has been fine - tuned for a longer time with an additional 16 hours of data.

🚀 Quick Start

This model is designed for Finnish ASR. To see a detailed example of how to use it, check the run-finnish-asr-models.ipynb notebook in this repository.

✨ Features

• Fine - tuned for Finnish: Specifically optimized for Finnish ASR with a large amount of Finnish transcribed speech data.
• Included Language Model: Comes with a Finnish KenLM language model for the decoding phase.

📦 Installation

No installation steps were provided in the original README, so this section is skipped.

💻 Usage Examples

Basic Usage

To evaluate this model on the Common Voice 7.0 dataset, run the following command:

python3 eval.py --model_id Finnish-NLP/wav2vec2-xlsr-1b-finnish-lm  --dataset mozilla-foundation/common_voice_7_0 --config fi --split test

📚 Documentation

Model Description

Wav2Vec2 XLS - R is a large - scale multilingual pretrained model for speech developed by Facebook AI. It is pretrained on 436k hours of unlabeled speech from various sources such as VoxPopuli, MLS, CommonVoice, BABEL, and VoxLingua107. It uses the wav2vec 2.0 objective across 128 languages.

This model is a fine - tuned version of the pretrained model (1 billion parameter variant) for Finnish ASR. You can read more about the pretrained model from this blog and this paper.

Intended Uses & Limitations

Intended Use

You can use this model for Finnish ASR (speech - to - text) tasks.

Limitations and Bias

• Audio Length: This model was fine - tuned with audio samples of maximum 20 seconds in length. It likely performs best on short audios of similar length, but you can also try it on longer audios. If you encounter out - of - memory errors with very long audio files, you can use the audio chunking method introduced in [this blog post](https://huggingface.co/blog/asr - chunking).
• Data Domain: A large portion of the fine - tuning data was from the Finnish Parliament dataset. So, the model may not generalize well to different domains like daily spoken Finnish with dialects.
• Gender and Age Bias: The datasets' audios are mostly from adult males. Thus, the model may not work as well for children's and women's speeches.
• Language Model Generalization: The Finnish KenLM language model used in the decoding phase was trained with text data from audio transcriptions. It may not generalize well to different language styles, such as daily spoken language with dialects. It might be beneficial to train your own KenLM language model for your specific domain.

Training Data

This model was fine - tuned with 259.57 hours of Finnish transcribed speech data from the following datasets:

The datasets were filtered to include audio samples with a maximum length of 20 seconds.

Training Procedure

This model was trained during the [Robust Speech Challenge Event](https://discuss.huggingface.co/t/open - to - the - community - robust - speech - recognition - challenge/13614) organized by Hugging Face. The training was conducted on a Tesla V100 GPU sponsored by OVHcloud.

The training script was provided by Hugging Face and is available [here](https://github.com/huggingface/transformers/blob/main/examples/research_projects/robust - speech - event/run_speech_recognition_ctc_bnb.py). Only the data loading was modified for custom datasets.

For the KenLM language model training, the [blog post tutorial](https://huggingface.co/blog/wav2vec2 - with - ngram) provided by Hugging Face was followed. The training data for the 5 - gram KenLM was the text transcriptions of the audio training data.

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: 8 - bit Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5
• mixed_precision_training: Native AMP

The pretrained facebook/wav2vec2-xls-r-1b model was initialized with the following hyperparameters:

• attention_dropout: 0.094
• hidden_dropout: 0.047
• feat_proj_dropout: 0.04
• mask_time_prob: 0.082
• layerdrop: 0.041
• activation_dropout: 0.055
• ctc_loss_reduction: "mean"

Training Results

Framework Versions

• Transformers 4.17.0.dev0
• Pytorch 1.10.2+cu102
• Datasets 1.18.3
• Tokenizers 0.11.0

Evaluation Results

Evaluation was performed on the Common Voice 7.0 Finnish test split, Common Voice 9.0 Finnish test split, and the FLEURS ASR Finnish test split.

This model's training data includes the training splits of Common Voice 7.0, while newer models Finnish-NLP/wav2vec2-base-fi-voxpopuli-v2-finetuned and Finnish-NLP/wav2vec2-large-uralic-voxpopuli-v2-finnish include Common Voice 9.0. Tests were run for both Common Voice versions. Note that Common Voice may not fully preserve the test split between dataset versions, so comparisons between models trained with different Common Voice versions are not entirely accurate but still meaningful.

Common Voice 7.0 Testing

To evaluate this model, run the eval.py script in this repository:

python3 eval.py --model_id Finnish-NLP/wav2vec2-xlsr-1b-finnish-lm  --dataset mozilla-foundation/common_voice_7_0 --config fi --split test

This model (the fourth row of the table) achieves the following WER (Word Error Rate) and CER (Character Error Rate) results compared to other models and their parameter counts:

Common Voice 9.0 Testing

To evaluate this model, run the eval.py script in this repository:

python3 eval.py --model_id Finnish-NLP/wav2vec2-xlsr-1b-finnish-lm  --dataset mozilla-foundation/common_voice_9_0 --config fi --split test

This model (the fourth row of the table) achieves the following WER (Word Error Rate) and CER (Character Error Rate) results compared to other models and their parameter counts:

FLEURS ASR Testing

The evaluation command for the FLEURS ASR dataset was not fully provided in the original README.

🔧 Technical Details

Model Architecture

The model is based on the Wav2Vec2 XLS - R architecture, which is a powerful multilingual speech model. The fine - tuning process adapts this general - purpose architecture to the Finnish language for ASR tasks.

Training Process

The fine - tuning was carried out with a large amount of Finnish transcribed speech data. The use of a Tesla V100 GPU and specific hyperparameters ensured efficient training. The KenLM language model was trained separately using text transcriptions of the audio training data.

📄 License

This model is licensed under the Apache 2.0 license.