Wav2vec2 Large Xlsr Lithuanian
This is a Lithuanian automatic speech recognition (ASR) model fine-tuned from Facebook's wav2vec2-large-xlsr-53 model, trained using the Common Voice dataset.
Downloads 570
Release Time : 3/2/2022
Model Overview
This model is specifically designed for Lithuanian automatic speech recognition tasks, capable of converting Lithuanian speech into text.
Model Features
High Accuracy Lithuanian Recognition
A speech recognition model optimized specifically for Lithuanian, achieving 34.66% WER on the Common Voice test set
Based on XLSR Architecture
Uses Facebook's wav2vec2-large-xlsr-53 as the base model, featuring strong cross-lingual representation capabilities
No Language Model Required
Can be used directly without additional language model support
Model Capabilities
Lithuanian speech recognition
16kHz audio processing
End-to-end speech-to-text
Use Cases
Speech Transcription
Speech Content Transcription
Automatically converts Lithuanian speech content into text
34.66% WER on the Common Voice test set
Voice Assistants
Lithuanian Voice Command Recognition
Used for front-end speech recognition in Lithuanian voice assistants
🚀 Wav2Vec2-Large-XLSR-53-Lithuanian
This model is a fine - tuned version of facebook/wav2vec2-large-xlsr-53 in Lithuanian using Common Voice. Ensure your speech input is sampled at 16kHz when using this model.
🚀 Quick Start
The model can be used directly (without a language model) as follows:
📦 Installation
Requirements
# requirement packages
!pip install git+https://github.com/huggingface/datasets.git
!pip install git+https://github.com/huggingface/transformers.git
!pip install torchaudio
!pip install librosa
!pip install jiwer
Normalizer
!wget -O normalizer.py https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-lithuanian/raw/main/normalizer.py
💻 Usage Examples
Basic Usage
import librosa
import torch
import torchaudio
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
from datasets import load_dataset
import numpy as np
import re
import string
import IPython.display as ipd
from normalizer import normalizer
def speech_file_to_array_fn(batch):
speech_array, sampling_rate = torchaudio.load(batch["path"])
speech_array = speech_array.squeeze().numpy()
speech_array = librosa.resample(np.asarray(speech_array), sampling_rate, 16_000)
batch["speech"] = speech_array
return batch
def predict(batch):
features = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
input_values = features.input_values.to(device)
attention_mask = features.attention_mask.to(device)
with torch.no_grad():
logits = model(input_values, attention_mask=attention_mask).logits
pred_ids = torch.argmax(logits, dim=-1)
batch["predicted"] = processor.batch_decode(pred_ids)[0]
return batch
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
processor = Wav2Vec2Processor.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-lithuanian")
model = Wav2Vec2ForCTC.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-lithuanian").to(device)
dataset = load_dataset("common_voice", "lt", split="test[:1%]")
dataset = dataset.map(
normalizer,
fn_kwargs={"remove_extra_space": True},
remove_columns=list(set(dataset.column_names) - set(['sentence', 'path']))
)
dataset = dataset.map(speech_file_to_array_fn)
result = dataset.map(predict)
max_items = np.random.randint(0, len(result), 20).tolist()
for i in max_items:
reference, predicted = result["sentence"][i], result["predicted"][i]
print("reference:", reference)
print("predicted:", predicted)
print('---')
Output:
reference: jos tikslas buvo rasti kelią į ramųjį vandenyną šiaurės amerikoje
predicted: jos tikstas buvo rasikelia į ramų į vandenyna šiaurės amerikoje
---
reference: pietrytinėje dalyje likusių katalikų kapinių teritorija po antrojo pasaulinio karo dar padidėjo
predicted: pietrytinė daljelikusių gatalikų kapinių teritoriją pontro pasaulnio karo dar padidėjo
---
reference: koplyčioje pakabintas aušros vartų marijos paveikslas
predicted: koplyčioje pakagintas aušos fortų marijos paveikslas
---
reference: yra politinių debatų vedėjas
predicted: yra politinių debatų vedėjas
---
reference: žmogui taip pat gali būti mirtinai pavojingi
predicted: žmogui taip pat gali būti mirtinai pavojingi
---
reference: tuo pačiu metu kijeve nuverstas netekęs vokietijos paramos skoropadskis
predicted: tuo pačiu metu kiei venų verstas netekės vokietijos paramos kropadskis
---
reference: visos dvylika komandų tarpusavyje sužaidžia po dvi rungtynes
predicted: visos dvylika komandų tarpuso vysų žaidžia po dvi rungtynės
---
reference: kaukazo regioną sudaro kaukazo kalnai ir gretimos žemumos
predicted: kau kazo regioną sudaro kaukazo kalnai ir gretimos žemumus
---
reference: tarptautinių ir rusiškų šaškių kandidatas į sporto meistrus
predicted: tarptautinio ir rusiškos šaškių kandidatus į sporto meistrus
---
reference: prasideda putorano plynaukštės pietiniame pakraštyje
predicted: prasideda futorano prynaukštės pietiniame pakraštyje
---
reference: miestas skirstomas į senamiestį ir naujamiestį
predicted: miestas skirstomas į senamėsti ir naujamiestė
---
reference: tais pačiais metais pelnė bronzą pasaulio taurės kolumbijos etape komandinio sprinto rungtyje
predicted: tais pačiais metais pelnį mronsa pasaulio taurės kolumbijos etape komandinio sprento rungtyje
---
reference: prasideda putorano plynaukštės pietiniame pakraštyje
predicted: prasideda futorano prynaukštės pietiniame pakraštyje
---
reference: moterų tarptautinės meistrės vardas yra viena pakopa žemesnis už moterų tarptautinės korespondencinių šachmatų didmeistrės
predicted: moterų tarptautinės meistrės vardas yra gana pakopo žymesnis už moterų tarptautinės kūrespondencinių šachmatų didmesčias
---
reference: teritoriją dengia tropinės džiunglės
predicted: teritorija dengia tropinės žiunglės
---
reference: pastaroji dažnai pereina į nimcovičiaus gynybą arba bogoliubovo gynybą
predicted: pastaruoji dažnai pereina nimcovičiaus gynyba arba bogalių buvo gymyba
---
reference: už tai buvo suimtas ir tris mėnesius sėdėjo butyrkų kalėjime
predicted: užtai buvo sujumtas ir tris mėne susiedėjo butirkų kalėjime
---
reference: tai didžiausias pagal gyventojų skaičių regionas
predicted: tai didžiausias pagal gyventojų skaičių redionus
---
reference: vilkyškių miške taip pat auga raganų eglė
predicted: vilkiškimiškė taip pat auga ragano eglė
---
reference: kitas gavo skaraitiškės dvarą su palivarkais
predicted: kitas gavos karaitiškės dvarą spolivarkais
---
🔧 Evaluation
The model can be evaluated as follows on the test data of Common Voice.
import librosa
import torch
import torchaudio
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
from datasets import load_dataset, load_metric
import numpy as np
import re
import string
from normalizer import normalizer
def speech_file_to_array_fn(batch):
speech_array, sampling_rate = torchaudio.load(batch["path"])
speech_array = speech_array.squeeze().numpy()
speech_array = librosa.resample(np.asarray(speech_array), sampling_rate, 16_000)
batch["speech"] = speech_array
return batch
def predict(batch):
features = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
input_values = features.input_values.to(device)
attention_mask = features.attention_mask.to(device)
with torch.no_grad():
logits = model(input_values, attention_mask=attention_mask).logits
pred_ids = torch.argmax(logits, dim=-1)
batch["predicted"] = processor.batch_decode(pred_ids)[0]
return batch
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
processor = Wav2Vec2Processor.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-lithuanian")
model = Wav2Vec2ForCTC.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-lithuanian").to(device)
dataset = load_dataset("common_voice", "lt", split="test")
dataset = dataset.map(
normalizer,
fn_kwargs={"remove_extra_space": True},
remove_columns=list(set(dataset.column_names) - set(['sentence', 'path']))
)
dataset = dataset.map(speech_file_to_array_fn)
result = dataset.map(predict)
wer = load_metric("wer")
print("WER: {:.2f}".format(100 * wer.compute(predictions=result["predicted"], references=result["sentence"])))
Test Result:
- WER: 34.66%
📚 Training & Report
The Common Voice train
, validation
datasets were used for training.
You can see the training states here
The script used for training can be found here
❓ Questions?
Post a Github issue on the Wav2Vec repo.
📄 License
This project is licensed under the apache - 2.0
license.
📋 Information Table
Property | Details |
---|---|
Model Type | Fine - tuned facebook/wav2vec2-large-xlsr-53 in Lithuanian |
Training Data | Common Voice train , validation datasets |
Tags | audio, automatic - speech - recognition, speech, xlsr - fine - tuning - week |
Widget Examples | Common Voice sample 11, Common Voice sample 74 |
Model Index | Name: XLSR Wav2Vec2 Lithuanian by Mehrdad Farahani, Task: Speech Recognition, Dataset: Common Voice lt, Metrics: Test WER = 34.66% |
Voice Activity Detection
MIT
Voice activity detection model based on pyannote.audio 2.1, used to identify speech activity segments in audio
Speech Recognition
V
pyannote
7.7M
181
Wav2vec2 Large Xlsr 53 Portuguese
Apache-2.0
This is a fine-tuned XLSR-53 large model for Portuguese speech recognition tasks, trained on the Common Voice 6.1 dataset, supporting Portuguese speech-to-text conversion.
Speech Recognition Other
W
jonatasgrosman
4.9M
32
Whisper Large V3
Apache-2.0
Whisper is an advanced automatic speech recognition (ASR) and speech translation model proposed by OpenAI, trained on over 5 million hours of labeled data, with strong cross-dataset and cross-domain generalization capabilities.
Speech Recognition Supports Multiple Languages
W
openai
4.6M
4,321
Whisper Large V3 Turbo
MIT
Whisper is a state-of-the-art automatic speech recognition (ASR) and speech translation model developed by OpenAI, trained on over 5 million hours of labeled data, demonstrating strong generalization capabilities in zero-shot settings.
Speech Recognition
Transformers Supports Multiple Languages

W
openai
4.0M
2,317
Wav2vec2 Large Xlsr 53 Russian
Apache-2.0
A Russian speech recognition model fine-tuned from facebook/wav2vec2-large-xlsr-53, supporting 16kHz sampled audio input
Speech Recognition Other
W
jonatasgrosman
3.9M
54
Wav2vec2 Large Xlsr 53 Chinese Zh Cn
Apache-2.0
A Chinese speech recognition model fine-tuned based on facebook/wav2vec2-large-xlsr-53, supporting 16kHz sampling rate audio input.
Speech Recognition Chinese
W
jonatasgrosman
3.8M
110
Wav2vec2 Large Xlsr 53 Dutch
Apache-2.0
A Dutch speech recognition model fine-tuned based on facebook/wav2vec2-large-xlsr-53, trained on the Common Voice and CSS10 datasets, supporting 16kHz audio input.
Speech Recognition Other
W
jonatasgrosman
3.0M
12
Wav2vec2 Large Xlsr 53 Japanese
Apache-2.0
Japanese speech recognition model fine-tuned from facebook/wav2vec2-large-xlsr-53, supporting 16kHz sampling rate audio input
Speech Recognition Japanese
W
jonatasgrosman
2.9M
33
Mms 300m 1130 Forced Aligner
A text-to-audio forced alignment tool based on Hugging Face pre-trained models, supporting multiple languages with high memory efficiency
Speech Recognition
Transformers Supports Multiple Languages

M
MahmoudAshraf
2.5M
50
Wav2vec2 Large Xlsr 53 Arabic
Apache-2.0
Arabic speech recognition model fine-tuned from facebook/wav2vec2-large-xlsr-53, trained on Common Voice and Arabic speech corpus
Speech Recognition Arabic
W
jonatasgrosman
2.3M
37
Featured Recommended AI Models