Asr Wav2vec2 Ctc French

🚀 用于法语自动语音识别的微调wav2vec2 - FR - 7K - large模型

本模型专为解决法语自动语音识别问题而设计，基于大规模法语语音数据集进行微调，能有效提升语音识别的准确性和鲁棒性，在多种法语语音场景中具有出色的表现。

🚀 快速开始

本模型是 LeBenchmark/wav2vec2 - FR - 7K - large 的微调版本。它在包含超过2200小时法语语音音频的复合数据集上进行训练，这些数据集来自 Common Voice 11.0、Multilingual LibriSpeech、Voxpopuli、Multilingual TEDx、MediaSpeech 和 African Accented French 的训练集和验证集。使用该模型时，请确保输入的语音采样率为16Khz。

✨ 主要特性

• 微调优化：基于预训练模型进行微调，更适配法语语音识别任务。
• 多数据集训练：使用多个法语语音数据集训练，提升模型的泛化能力。
• 支持语言模型：可选择使用语言模型，进一步提高识别准确率。

📦 安装指南

文档中未提及安装步骤，若有需要可参考相关库（如 transformers、torch、torchaudio）的官方安装指南。

💻 使用示例

基础用法

使用语言模型处理本地音频文件

import torch
import torchaudio

from transformers import AutoModelForCTC, Wav2Vec2ProcessorWithLM

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model = AutoModelForCTC.from_pretrained("bhuang/asr-wav2vec2-french").to(device)
processor_with_lm = Wav2Vec2ProcessorWithLM.from_pretrained("bhuang/asr-wav2vec2-french")
model_sample_rate = processor_with_lm.feature_extractor.sampling_rate

wav_path = "example.wav"  # path to your audio file
waveform, sample_rate = torchaudio.load(wav_path)
waveform = waveform.squeeze(axis=0)  # mono

# resample
if sample_rate != model_sample_rate:
    resampler = torchaudio.transforms.Resample(sample_rate, model_sample_rate)
    waveform = resampler(waveform)

# normalize
input_dict = processor_with_lm(waveform, sampling_rate=model_sample_rate, return_tensors="pt")

with torch.inference_mode():
    logits = model(input_dict.input_values.to(device)).logits

predicted_sentence = processor_with_lm.batch_decode(logits.cpu().numpy()).text[0]

不使用语言模型处理本地音频文件

import torch
import torchaudio

from transformers import AutoModelForCTC, Wav2Vec2Processor

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model = AutoModelForCTC.from_pretrained("bhuang/asr-wav2vec2-french").to(device)
processor = Wav2Vec2Processor.from_pretrained("bhuang/asr-wav2vec2-french")
model_sample_rate = processor.feature_extractor.sampling_rate

wav_path = "example.wav"  # path to your audio file
waveform, sample_rate = torchaudio.load(wav_path)
waveform = waveform.squeeze(axis=0)  # mono

# resample
if sample_rate != model_sample_rate:
    resampler = torchaudio.transforms.Resample(sample_rate, model_sample_rate)
    waveform = resampler(waveform)

# normalize
input_dict = processor(waveform, sampling_rate=model_sample_rate, return_tensors="pt")

with torch.inference_mode():
    logits = model(input_dict.input_values.to(device)).logits

# decode
predicted_ids = torch.argmax(logits, dim=-1)
predicted_sentence = processor.batch_decode(predicted_ids)[0]

📚 详细文档

评估

在 mozilla - foundation/common_voice_11_0 上进行评估

python eval.py \
  --model_id "bhuang/asr-wav2vec2-french" \
  --dataset "mozilla-foundation/common_voice_11_0" \
  --config "fr" \
  --split "test" \
  --log_outputs \
  --outdir "outputs/results_mozilla-foundatio_common_voice_11_0_with_lm"

在 speech - recognition - community - v2/dev_data 上进行评估

python eval.py \
  --model_id "bhuang/asr-wav2vec2-french" \
  --dataset "speech-recognition-community-v2/dev_data" \
  --config "fr" \
  --split "validation" \
  --chunk_length_s 30.0 \
  --stride_length_s 5.0 \
  --log_outputs \
  --outdir "outputs/results_speech-recognition-community-v2_dev_data_with_lm"

模型指标

评估结果

📄 许可证

本模型采用Apache 2.0许可证。