Wav2vec2 Xls R Hindi

🚀 shivam/wav2vec2-xls-r-hindi

This model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the MOZILLA - FOUNDATION/COMMON_VOICE_7_0 - HI dataset. It is designed for automatic speech recognition tasks, aiming to accurately transcribe Hindi speech.

🚀 Quick Start

This model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the MOZILLA - FOUNDATION/COMMON_VOICE_7_0 - HI dataset. It achieves the following results on the evaluation set:

• Loss: 1.2282
• Wer: 0.6838

💻 Usage Examples

Basic Usage

The basic usage of this model involves loading it and using it for speech recognition tasks. Here is a simple example code snippet (assuming you have the necessary libraries installed):

# Load the model
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import torch
from datasets import load_dataset
import soundfile as sf

model = Wav2Vec2ForCTC.from_pretrained("shivam/wav2vec2-xls-r-hindi")
processor = Wav2Vec2Processor.from_pretrained("shivam/wav2vec2-xls-r-hindi")

# Load an audio file
def map_to_array(batch):
    speech, _ = sf.read(batch["path"])
    batch["speech"] = speech
    return batch

dataset = load_dataset("mozilla-foundation/common_voice_7_0", "hi", split="test[:10]")
dataset = dataset.map(map_to_array)

# Prepare the input
input_values = processor(dataset["speech"][0], return_tensors="pt").input_values

# Inference
with torch.no_grad():
    logits = model(input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.decode(predicted_ids[0])
print(transcription)

📚 Documentation

Evaluation results on Common Voice 7 "test" (Running ./eval.py)

With LM

• WER: 52.30
• CER: 26.09

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 7.5e - 05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2000
• num_epochs: 50.0
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.16.0.dev0
• Pytorch 1.10.1+cu113
• Datasets 1.18.1.dev0
• Tokenizers 0.11.0

📄 License

This model is licensed under the Apache - 2.0 license.

📦 Model Information