Wav2vec2 Base 960h 4 Gram

🚀 Wav2Vec2-Base-960h + 4-gram

This model is designed for automatic speech recognition. It's identical to Facebook's Wav2Vec2-Base-960h, but enhanced with an English 4-gram, using the 4-gram.arpa.gz from Librispeech's official ngrams.

📦 Information

💻 Usage Examples

Basic Usage

from datasets import load_dataset
from transformers import AutoModelForCTC, AutoProcessor
import torch
from jiwer import wer

model_id = "patrickvonplaten/wav2vec2-base-960h-4-gram"

librispeech_eval = load_dataset("librispeech_asr", "other", split="test")

model = AutoModelForCTC.from_pretrained(model_id).to("cuda")
processor = AutoProcessor.from_pretrained(model_id)

def map_to_pred(batch):
    inputs = processor(batch["audio"]["array"], sampling_rate=16_000, return_tensors="pt")

    inputs = {k: v.to("cuda") for k,v in inputs.items()}

    with torch.no_grad():
        logits = model(**inputs).logits

    transcription = processor.batch_decode(logits.cpu().numpy()).text[0]
    batch["transcription"] = transcription
    return batch

result = librispeech_eval.map(map_to_pred, remove_columns=["audio"])

print(wer(result["text"], result["transcription"]))

Result (WER)

📚 Documentation

Model Index

• Name: patrickvonplaten/wav2vec2-base-960h-4-gram
• Results:
  • Task: Automatic Speech Recognition
    • Dataset: LibriSpeech (clean)
      • Type: librispeech_asr
      • Config: clean
      • Split: test
      • Args:
        • Language: en
    • Metrics:
      • Name: Test WER
      • Type: wer
      • Value: 2.59
  • Task: Automatic Speech Recognition
    • Dataset: LibriSpeech (other)
      • Type: librispeech_asr
      • Config: other
      • Split: test
      • Args:
        • Language: en
    • Metrics:
      • Name: Test WER
      • Type: wer
      • Value: 6.46

📄 License

This project is licensed under the apache-2.0 license.