Open-source model wav2vec2-large-xlsr-53-spanish-with-lm - Freely improve the accuracy of Spanish speech recognition!

Wav2vec2 Large Xlsr 53 Spanish With Lm

Developed by patrickvonplaten

This is a Spanish speech recognition model based on Wav2Vec2-Large-XLSR-53, enhanced with language model support, significantly improving recognition accuracy.

Speech Recognition SpanishOpen Source License:Apache-2.0 #Spanish speech recognition #Low word error rate #Language model enhanced

Downloads 2,198

Release Time : 3/2/2022

Model Overview

This model is an improved version of jonatasgrosman/wav2vec2-large-xlsr-53-spanish, which increases the accuracy of Spanish speech recognition by integrating language model support.

Model Features

Language Model Integration

Integrates language model through pyctcdecode library, significantly improving recognition accuracy

Performance Improvement

Compared to the original model, WER is reduced by 20% on the Spanish Common Voice test set

Ease of Use

Provides a processor that seamlessly integrates with Transformers, with usage similar to the original model

Model Capabilities

Spanish speech recognition

Audio to text conversion

Large-scale speech processing

Use Cases

Speech Transcription

Spanish Speech Transcription

Convert Spanish speech content into text

Achieves 8.44% WER on the Common Voice test set

Voice Assistants

Spanish Voice Command Recognition

Used for front-end speech recognition in Spanish voice assistants

🚀 Wav2Vec2-Large-XLSR-53-Spanish-With-LM

This is a model that copies Wav2Vec2-Large-XLSR-53-Spanish and supports language models, aiming to enhance the performance of automatic speech recognition.

🚀 Quick Start

This model card can be seen as a demo for the pyctcdecode integration with Transformers led by this PR. The PR explains in - detail how the integration works.

In a nutshell: This PR adds a new Wav2Vec2WithLMProcessor class as a drop - in replacement for Wav2Vec2Processor.

The only change from the existing ASR pipeline will be:

✨ Features

Changes

import torch
from datasets import load_dataset
from transformers import AutoModelForCTC, AutoProcessor
import torchaudio.functional as F


model_id = "patrickvonplaten/wav2vec2-large-xlsr-53-spanish-with-lm"

sample = next(iter(load_dataset("common_voice", "es", split="test", streaming=True)))
resampled_audio = F.resample(torch.tensor(sample["audio"]["array"]), 48_000, 16_000).numpy()

model = AutoModelForCTC.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id)

input_values = processor(resampled_audio, return_tensors="pt").input_values

with torch.no_grad():
    logits = model(input_values).logits

-prediction_ids = torch.argmax(logits, dim=-1)
-transcription = processor.batch_decode(prediction_ids)
+transcription = processor.batch_decode(logits.numpy()).text
# => 'bien y qué regalo vas a abrir primero'

Improvement

This model has been compared on 512 speech samples from the Spanish Common Voice Test set and gives a nice 20 % performance boost:

The results can be reproduced by running from this model repository:

Property	Details
Model Type	Wav2Vec2-Large-XLSR-53-Spanish-With-LM
Training Data	common_voice
Metrics	wer, cer
License	apache - 2.0

Model	WER	CER
patrickvonplaten/wav2vec2-large-xlsr-53-spanish-with-lm	8.44%	2.93%
jonatasgrosman/wav2vec2-large-xlsr-53-spanish	10.20%	3.24%

bash run_ngram_wav2vec2.py 1 512

bash run_ngram_wav2vec2.py 0 512

with run_ngram_wav2vec2.py being https://huggingface.co/patrickvonplaten/wav2vec2-large-xlsr-53-spanish-with-lm/blob/main/run_ngram_wav2vec2.py

📄 License

This project is licensed under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご