🚀 Wav2Vec2-Large-Ru-Golos
Wav2Vec2-Large-Ru-Golos 模型基於 facebook/wav2vec2-large-xlsr-53,使用 Sberdevices Golos 數據集針對俄語進行了微調,並採用了諸如音高變換、聲音加速/減速、混響等音頻增強技術。使用該模型時,請確保輸入語音的採樣率為 16kHz。
🚀 快速開始
本模型可作為獨立的聲學模型對音頻文件進行轉錄,以下是使用示例:
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
from datasets import load_dataset
import torch
processor = Wav2Vec2Processor.from_pretrained("bond005/wav2vec2-large-ru-golos")
model = Wav2Vec2ForCTC.from_pretrained("bond005/wav2vec2-large-ru-golos")
ds = load_dataset("bond005/sberdevices_golos_10h_crowd", split="test")
processed = processor(ds[0]["audio"]["array"], return_tensors="pt", padding="longest")
logits = model(processed.input_values, attention_mask=processed.attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)
✨ 主要特性
- 基於預訓練模型:基於 facebook/wav2vec2-large-xlsr-53 進行微調,繼承了強大的語音處理能力。
- 音頻增強:在微調過程中使用了音高變換、聲音加速/減速、混響等音頻增強技術,提高了模型的魯棒性。
- 多數據集支持:可在多個俄語語音數據集上進行訓練和評估,如 SberDevices/Golos、bond005/sova_rudevices、bond005/rulibrispeech 等。
💻 使用示例
基礎用法
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
from datasets import load_dataset
import torch
processor = Wav2Vec2Processor.from_pretrained("bond005/wav2vec2-large-ru-golos")
model = Wav2Vec2ForCTC.from_pretrained("bond005/wav2vec2-large-ru-golos")
ds = load_dataset("bond005/sberdevices_golos_10h_crowd", split="test")
processed = processor(ds[0]["audio"]["array"], return_tensors="pt", padding="longest")
logits = model(processed.input_values, attention_mask=processed.attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)
📚 詳細文檔
評估
以下代碼展示瞭如何在 Golos 數據集的 “crowd” 和 “farfield” 測試數據上評估 bond005/wav2vec2-large-ru-golos 模型:
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import torch
from jiwer import wer, cer
golos_crowd_test = load_dataset("bond005/sberdevices_golos_10h_crowd", split="test")
golos_crowd_test = golos_crowd_test.filter(
lambda it1: (it1["transcription"] is not None) and (len(it1["transcription"].strip()) > 0)
)
golos_farfield_test = load_dataset("bond005/sberdevices_golos_100h_farfield", split="test")
golos_farfield_test = golos_farfield_test.filter(
lambda it2: (it2["transcription"] is not None) and (len(it2["transcription"].strip()) > 0)
)
model = Wav2Vec2ForCTC.from_pretrained("facebook/wav2vec2-base-960h").to("cuda")
processor = Wav2Vec2Processor.from_pretrained("facebook/wav2vec2-base-960h")
def map_to_pred(batch):
processed = processor(
batch["audio"]["array"], sampling_rate=batch["audio"]["sampling_rate"],
return_tensors="pt", padding="longest"
)
input_values = processed.input_values.to("cuda")
attention_mask = processed.attention_mask.to("cuda")
with torch.no_grad():
logits = model(input_values, attention_mask=attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)
batch["text"] = transcription[0]
return batch
crowd_result = golos_crowd_test.map(map_to_pred, remove_columns=["audio"])
crowd_wer = wer(crowd_result["transcription"], crowd_result["text"])
crowd_cer = cer(crowd_result["transcription"], crowd_result["text"])
print("Word error rate on the Crowd domain:", crowd_wer)
print("Character error rate on the Crowd domain:", crowd_cer)
farfield_result = golos_farfield_test.map(map_to_pred, remove_columns=["audio"])
farfield_wer = wer(farfield_result["transcription"], farfield_result["text"])
farfield_cer = cer(farfield_result["transcription"], farfield_result["text"])
print("Word error rate on the Farfield domain:", farfield_wer)
print("Character error rate on the Farfield domain:", farfield_cer)
結果 (WER, %):
"crowd" |
"farfield" |
10.144 |
20.353 |
結果 (CER, %):
"crowd" |
"farfield" |
2.168 |
6.030 |
你可以在我的 Kaggle 網頁 https://www.kaggle.com/code/bond005/wav2vec2-ru-eval 上查看該模型在其他數據集(包括 Russian Librispeech 和 SOVA RuDevices)上的評估腳本。
📄 許可證
本項目採用 Apache-2.0 許可證。
📚 詳細信息
數據集
屬性 |
詳情 |
訓練數據集 |
SberDevices/Golos、bond005/sova_rudevices、bond005/rulibrispeech |
評估指標 |
單詞錯誤率 (WER)、字符錯誤率 (CER) |
模型索引
- 模型名稱:XLSR Wav2Vec2 Russian by Ivan Bondarenko
- 評估結果:
- 任務:語音識別(Automatic Speech Recognition)
- 數據集:Sberdevices Golos (crowd)、Sberdevices Golos (farfield)、Common Voice ru、Sova RuDevices、Russian Librispeech、Voxforge Ru
- 評估指標:單詞錯誤率 (WER)、字符錯誤率 (CER)
引用
如果你想引用此模型,可以使用以下 BibTeX 格式:
@misc{bondarenko2022wav2vec2-large-ru-golos,
title={XLSR Wav2Vec2 Russian by Ivan Bondarenko},
author={Bondarenko, Ivan},
publisher={Hugging Face},
journal={Hugging Face Hub},
howpublished={\url{https://huggingface.co/bond005/wav2vec2-large-ru-golos}},
year={2022}
}