Roest Wav2vec2 1B V2

🚀 Røst-wav2vec2-1B-v2

This is a Danish state-of-the-art speech recognition model, trained as part of the CoRal project by Alvenir. It aims to provide high - quality automatic speech recognition for the Danish language.

🚀 Quick Start

Start by installing the required libraries:

$ pip install transformers kenlm pyctcdecode

Next you can use the model using the transformers Python package as follows:

>>> from transformers import pipeline
>>> audio = get_audio()  # 16kHz raw audio array
>>> transcriber = pipeline(model="CoRal-project/roest-wav2vec2-1B-v2")
>>> transcriber(audio)
{'text': 'your transcription'}

✨ Features

• Trained on the CoRal-v2 dataset, which includes rich Danish conversational and read - aloud data across diverse age groups, genders, and dialects.
• Based on the pre - trained wav2vec2 - xls - r - 1b model, fine - tuned for better performance in Danish speech recognition.

📦 Installation

To use this model, you need to install the necessary libraries:

$ pip install transformers kenlm pyctcdecode

💻 Usage Examples

Basic Usage

from transformers import pipeline
audio = get_audio()  # 16kHz raw audio array
transcriber = pipeline(model="CoRal-project/roest-wav2vec2-1B-v2")
transcription = transcriber(audio)
print(transcription)

📚 Documentation

Model Details

Wav2Vec2 is a state - of - the - art model architecture for speech recognition, leveraging self - supervised learning from raw audio data. The pre - trained wav2vec2 - xls - r - 1b has been fine - tuned for automatic speech recognition with the CoRal - v2 dataset dataset to enhance its performance in recognizing Danish speech with consideration to different dialects. The model was trained for 30K steps using the training setup in the CoRaL repository by running:

python src/scripts/finetune_asr_model.py  \
model=wav2vec2-medium \
max_steps=30000 \
datasets.coral_conversation_internal.id=CoRal-project/coral-v2 \
datasets.coral_readaloud_internal.id=CoRal-project/coral-v2

The model is evaluated using a Language Model (LM) as post - processing. The utilized LM is the one trained and used by CoRal - project/roest - wav2vec2 - 315m - v1. The model was trained on the CoRal - v2 dataset, including both the conversational and read - aloud subset. This dataset consists of Danish speech across a variety of dialects, age groups and gender distinctions. Note that the dataset used is licensed under a custom license, adapted from OpenRAIL - M, which allows commercial use with a few restrictions (speech synthesis and biometric identification). See license.

Evaluation

The model was evaluated using the following metrics:

• Character Error Rate (CER): The percentage of characters incorrectly transcribed.
• Word Error Rate (WER): The percentage of words incorrectly transcribed.

Conversational CoRal Performance

The model was firstly evaluated on a tentative version of the coral - v2 conversation dataset. The results are tentative as the test set only includes 5 unique speakers, of which 4 are women. The test set includes 2 speakers with 'Fynsk' dialect, 1 with 'Sønderjysk', 1 with 'Non - native' and 1 'Nordjysk'. Note that the high generalization error on conversation data for models trained on read - aloud data is still being analyzed.

Read - aloud CoRal Performance

OBS! Benchmark for hviske - v2 has been re - evaluated and the confidence interval is larger than reported in the model card.

📄 License

The model is licensed under the OpenRAIL license. See license for more details.