Wav2vec2 Base 10k Voxpopuli Ft Ro

🚀 Wav2Vec2-Base-VoxPopuli-Finetuned

This project is based on Facebook's Wav2Vec2, a base model. It was pretrained on the 10K unlabeled subset of the VoxPopuli corpus and fine - tuned on the transcribed Romanian data (refer to Table 1 of the paper for more details).

✨ Features

• Audio Processing: Specialized for audio data, especially in the field of automatic speech recognition.
• Multilingual Adaptability: Based on a large - scale multilingual speech corpus, VoxPopuli.

📄 License

This project is licensed under the CC - BY - NC - 4.0 license.

📚 Documentation

Paper

VoxPopuli: A Large - Scale Multilingual Speech Corpus for Representation Learning, Semi - Supervised Learning and Interpretation

Authors

Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, Emmanuel Dupoux from Facebook AI

For more information, please visit the official website here

🚀 Quick Start

Prerequisites

• Make sure you have installed the necessary libraries such as transformers, datasets, torchaudio, and torch.

Installation

The model can be easily installed using the transformers library.

💻 Usage Examples

Basic Usage

#!/usr/bin/env python3
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
from datasets import load_dataset
import torchaudio
import torch

# resample audio

# load model & processor
model = Wav2Vec2ForCTC.from_pretrained("facebook/wav2vec2-base-10k-voxpopuli-ft-ro")
processor = Wav2Vec2Processor.from_pretrained("facebook/wav2vec2-base-10k-voxpopuli-ft-ro")

# load dataset
ds = load_dataset("common_voice", "ro", split="validation[:1%]")

# common voice does not match target sampling rate
common_voice_sample_rate = 48000
target_sample_rate = 16000

resampler = torchaudio.transforms.Resample(common_voice_sample_rate, target_sample_rate)


# define mapping fn to read in sound file and resample
def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    speech = resampler(speech)
    batch["speech"] = speech[0]
    return batch


# load all audio files
ds = ds.map(map_to_array)

# run inference on the first 5 data samples
inputs = processor(ds[:5]["speech"], sampling_rate=target_sample_rate, return_tensors="pt", padding=True)

# inference
logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, axis=-1)

print(processor.batch_decode(predicted_ids))

This code demonstrates how to use the model for inference on a sample of the Common Voice dataset.