Wav2vec2 Large Robust Ft Swbd 300h

🚀 Wav2Vec2-Large-Robust finetuned on Switchboard

This is a fine - tuned version of the Wav2Vec2 - Large - Robust model, suitable for speech recognition tasks with various audio data sources.

🚀 Quick Start

This model is based on Facebook's Wav2Vec2. It's a fine - tuned version of the wav2vec2 - large - robust model.

Datasets

The model has been pretrained on the following datasets:

• Libri - Light: Open - source audio books from the LibriVox project; clean, read - out audio data.
• CommonVoice: Crowd - source collected audio data; read - out text snippets.
• Switchboard: Telephone speech corpus; noisy telephone data.
• Fisher: Conversational telephone speech; noisy telephone data.

It has subsequently been finetuned on 300 hours of Switchboard data.

Note

When using the model, make sure that your speech input is also sampled at 16Khz.

Paper

Paper Robust Wav2Vec2

Authors

Wei - Ning Hsu, Anuroop Sriram, Alexei Baevski, Tatiana Likhomanenko, Qiantong Xu, Vineel Pratap, Jacob Kahn, Ann Lee, Ronan Collobert, Gabriel Synnaeve, Michael Auli

Abstract

Self - supervised learning of speech representations has been a very active research area but most work is focused on a single domain such as read audio books for which there exist large quantities of labeled and unlabeled data. In this paper, we explore more general setups where the domain of the unlabeled data for pre - training data differs from the domain of the labeled data for fine - tuning, which in turn may differ from the test data domain. Our experiments show that using target domain data during pre - training leads to large performance improvements across a variety of setups. On a large - scale competitive setup, we show that pre - training on unlabeled in - domain data reduces the gap between models trained on in - domain and out - of - domain labeled data by 66% - 73%. This has obvious practical implications since it is much easier to obtain unlabeled target domain data than labeled data. Moreover, we find that pre - training on multiple domains improves generalization performance on domains not seen during training. Code and models will be made available at this https URL.

Original Model

The original model can be found under https://github.com/pytorch/fairseq/tree/master/examples/wav2vec#wav2vec-20.

✨ Features

• Multiple Datasets: Trained on a variety of datasets including clean and noisy audio, enhancing its adaptability.
• Fine - Tuned: Finetuned on 300 hours of Switchboard data for better performance on telephone speech.

📦 Installation

No specific installation steps are provided in the original README. If you want to use the model, you need to install the necessary Python libraries such as transformers and datasets. You can install them using pip:

pip install transformers datasets torch

💻 Usage Examples

Basic Usage

To transcribe audio files, the model can be used as a standalone acoustic model as follows:

from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
from datasets import load_dataset
import torch

# load model and processor
processor = Wav2Vec2Processor.from_pretrained("facebook/wav2vec2-large-robust-ft-swbd-300h")
model = Wav2Vec2ForCTC.from_pretrained("facebook/wav2vec2-large-robust-ft-swbd-300h")

# load dummy dataset and read soundfiles
ds = load_dataset("patrickvonplaten/librispeech_asr_dummy", "clean", split="validation")

# tokenize
input_values = processor(ds[0]["audio"]["array"], return_tensors="pt", padding="longest").input_values  # Batch size 1

# retrieve logits
logits = model(input_values).logits

# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)

📚 Documentation

Datasets

Widget

• Example 1: Librispeech sample 1
• Example 2: Librispeech sample 2

📄 License

This model is licensed under the apache - 2.0 license.