Wav2vec2 Xls R Parlaspeech Hr

🚀 wav2vec2-xls-r-parlaspeech-hr

This model is designed for Croatian Automatic Speech Recognition (ASR), leveraging the power of deep learning to accurately transcribe Croatian speech.

🚀 Quick Start

This Croatian ASR model is built upon the facebook/wav2vec2-xls-r-300m model. It was fine - tuned using 300 hours of recordings and transcripts from the Croatian parliament ASR dataset ParlaSpeech - HR v1.0.

If you utilize this model, please cite the following paper:

Nikola Ljubešić, Danijel Koržinek, Peter Rupnik, Ivo - Pavao Jazbec. ParlaSpeech - HR -- a freely available ASR dataset for Croatian bootstrapped from the ParlaMint corpus. http://www.lrec-conf.org/proceedings/lrec2022/workshops/ParlaCLARINIII/pdf/2022.parlaclariniii-1.16.pdf

✨ Features

• Dataset: Based on the Croatian parliament ASR dataset ParlaSpeech - HR v1.0.
• Model Base: Derived from facebook/wav2vec2-xls-r-300m.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
import soundfile as sf
import torch
import os

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

# load model and tokenizer
processor = Wav2Vec2Processor.from_pretrained(
    "classla/wav2vec2-xls-r-parlaspeech-hr")
model = Wav2Vec2ForCTC.from_pretrained("classla/wav2vec2-xls-r-parlaspeech-hr")


# download the example wav files:
os.system("wget https://huggingface.co/classla/wav2vec2-xls-r-parlaspeech-hr/raw/main/00020570a.flac.wav")

# read the wav file 
speech, sample_rate = sf.read("00020570a.flac.wav")
input_values = processor(speech, sampling_rate=sample_rate, return_tensors="pt").input_values.to(device)

# remove the raw wav file
os.system("rm 00020570a.flac.wav")

# retrieve logits
logits = model.to(device)(input_values).logits

# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.decode(predicted_ids[0]).lower()

# transcription: 'veliki broj poslovnih subjekata posluje sa minusom velik dio'

📚 Documentation

Metrics

Evaluation is carried out on the dev and test subsets of the ParlaSpeech - HR v1.0 dataset.

There are multiple models available. In terms of CER and WER, the best - performing model is wav2vec2-large-slavic-parlaspeech-hr-lm.

Training hyperparameters

During fine - tuning, the following arguments were employed:

Additional Information

• Language: Croatian
• Datasets: parlaspeech - hr
• Tags: audio, automatic - speech - recognition, parlaspeech
• Examples:
  • example 1
  • example 2