Asr Wav2vec2 Ctc French

🚀 用於法語自動語音識別的微調wav2vec2 - FR - 7K - large模型

本模型專為解決法語自動語音識別問題而設計，基於大規模法語語音數據集進行微調，能有效提升語音識別的準確性和魯棒性，在多種法語語音場景中具有出色的表現。

🚀 快速開始

本模型是 LeBenchmark/wav2vec2 - FR - 7K - large 的微調版本。它在包含超過2200小時法語語音音頻的複合數據集上進行訓練，這些數據集來自 Common Voice 11.0、Multilingual LibriSpeech、Voxpopuli、Multilingual TEDx、MediaSpeech 和 African Accented French 的訓練集和驗證集。使用該模型時，請確保輸入的語音採樣率為16Khz。

✨ 主要特性

• 微調優化：基於預訓練模型進行微調，更適配法語語音識別任務。
• 多數據集訓練：使用多個法語語音數據集訓練，提升模型的泛化能力。
• 支持語言模型：可選擇使用語言模型，進一步提高識別準確率。

📦 安裝指南

文檔中未提及安裝步驟，若有需要可參考相關庫（如 transformers、torch、torchaudio）的官方安裝指南。

💻 使用示例

基礎用法

使用語言模型處理本地音頻文件

import torch
import torchaudio

from transformers import AutoModelForCTC, Wav2Vec2ProcessorWithLM

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model = AutoModelForCTC.from_pretrained("bhuang/asr-wav2vec2-french").to(device)
processor_with_lm = Wav2Vec2ProcessorWithLM.from_pretrained("bhuang/asr-wav2vec2-french")
model_sample_rate = processor_with_lm.feature_extractor.sampling_rate

wav_path = "example.wav"  # path to your audio file
waveform, sample_rate = torchaudio.load(wav_path)
waveform = waveform.squeeze(axis=0)  # mono

# resample
if sample_rate != model_sample_rate:
    resampler = torchaudio.transforms.Resample(sample_rate, model_sample_rate)
    waveform = resampler(waveform)

# normalize
input_dict = processor_with_lm(waveform, sampling_rate=model_sample_rate, return_tensors="pt")

with torch.inference_mode():
    logits = model(input_dict.input_values.to(device)).logits

predicted_sentence = processor_with_lm.batch_decode(logits.cpu().numpy()).text[0]

不使用語言模型處理本地音頻文件

import torch
import torchaudio

from transformers import AutoModelForCTC, Wav2Vec2Processor

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model = AutoModelForCTC.from_pretrained("bhuang/asr-wav2vec2-french").to(device)
processor = Wav2Vec2Processor.from_pretrained("bhuang/asr-wav2vec2-french")
model_sample_rate = processor.feature_extractor.sampling_rate

wav_path = "example.wav"  # path to your audio file
waveform, sample_rate = torchaudio.load(wav_path)
waveform = waveform.squeeze(axis=0)  # mono

# resample
if sample_rate != model_sample_rate:
    resampler = torchaudio.transforms.Resample(sample_rate, model_sample_rate)
    waveform = resampler(waveform)

# normalize
input_dict = processor(waveform, sampling_rate=model_sample_rate, return_tensors="pt")

with torch.inference_mode():
    logits = model(input_dict.input_values.to(device)).logits

# decode
predicted_ids = torch.argmax(logits, dim=-1)
predicted_sentence = processor.batch_decode(predicted_ids)[0]

📚 詳細文檔

評估

在 mozilla - foundation/common_voice_11_0 上進行評估

python eval.py \
  --model_id "bhuang/asr-wav2vec2-french" \
  --dataset "mozilla-foundation/common_voice_11_0" \
  --config "fr" \
  --split "test" \
  --log_outputs \
  --outdir "outputs/results_mozilla-foundatio_common_voice_11_0_with_lm"

在 speech - recognition - community - v2/dev_data 上進行評估

python eval.py \
  --model_id "bhuang/asr-wav2vec2-french" \
  --dataset "speech-recognition-community-v2/dev_data" \
  --config "fr" \
  --split "validation" \
  --chunk_length_s 30.0 \
  --stride_length_s 5.0 \
  --log_outputs \
  --outdir "outputs/results_speech-recognition-community-v2_dev_data_with_lm"

模型指標

評估結果

📄 許可證

本模型採用Apache 2.0許可證。