Wav2vec2 Conformer Rope Large 100h Ft

🚀 Wav2Vec2-Conformer-Large-100h with Rotary Position Embeddings

Wav2Vec2 Conformer with rotary position embeddings, designed for automatic speech recognition. Pretrained on 960h of Librispeech and fine - tuned on 100 hours of Librispeech for 16kHz sampled speech audio.

🚀 Quick Start

This model is a Wav2Vec2 Conformer with rotary position embeddings. It has been pretrained on 960 hours of Librispeech and fine - tuned on 100 hours of Librispeech using 16kHz sampled speech audio. When using the model, ensure that your speech input is also sampled at 16kHz.

Paper: fairseq S2T: Fast Speech - to - Text Modeling with fairseq

Authors: Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Sravya Popuri, Dmytro Okhonko, Juan Pino

The results of Wav2Vec2 - Conformer can be found in Table 3 and Table 4 of the official paper.

The original model can be found under https://github.com/pytorch/fairseq/tree/master/examples/wav2vec#wav2vec - 20.

💻 Usage Examples

Basic Usage

from transformers import Wav2Vec2Processor, Wav2Vec2ConformerForCTC
from datasets import load_dataset
import torch

# load model and processor
processor = Wav2Vec2Processor.from_pretrained("facebook/wav2vec2-conformer-rope-large-100h-ft")
model = Wav2Vec2ConformerForCTC.from_pretrained("facebook/wav2vec2-conformer-rope-large-100h-ft")

# load dummy dataset and read soundfiles
ds = load_dataset("patrickvonplaten/librispeech_asr_dummy", "clean", split="validation")

# tokenize
input_values = processor(ds[0]["audio"]["array"], return_tensors="pt", padding="longest").input_values

# retrieve logits
logits = model(input_values).logits

# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)

📄 License

This project is licensed under the Apache - 2.0 license.