Wav2vec2 Large Xlsr 53 German With Lm

🚀 XLSR Wav2Vec2 德語模型

本項目的 XLSR Wav2Vec2 德語模型，可用於自動語音識別任務，在 Common Voice 德語測試數據上有出色表現。

🚀 快速開始

本模型可在 Common Voice 的德語測試數據上進行評估，具體評估方式見下方代碼示例。

✨ 主要特性

• 數據集：使用 Common Voice 數據集進行訓練。
• 評估指標：使用字錯率（WER）和字符錯誤率（CER）進行評估。
• 適用領域：適用於音頻、自動語音識別等領域。

📦 安裝指南

暫未提供相關安裝步驟。

💻 使用示例

基礎用法

以下代碼展示瞭如何在 Common Voice 的德語測試數據上評估該模型：

import torchaudio.functional as F
import torch
from transformers import AutoModelForCTC, AutoProcessor
import re
from datasets import load_dataset, load_metric

CHARS_TO_IGNORE = [",", "?", "¿", ".", "!", "¡", ";", "；", ":", '""', "%", '"', "�", "ʿ", "·", "჻", "~", "՞",
                   "؟", "،", "।", "॥", "«", "»", "„", "“", "”", "「", "」", "‘", "’", "《", "》", "(", ")", "[", "]",
                   "{", "}", "=", "`", "_", "+", "<", ">", "…", "–", "°", "´", "ʾ", "‹", "›", "©", "®", "—", "→", "。",
                   "、", "﹂", "﹁", "‧", "～", "﹏", "，", "｛", "｝", "（", "）", "［", "］", "【", "】", "‥", "〽",
                   "『', '』', '〝', '〟', '⟨', '⟩', '〜', '：', '！', '？', '♪', '؛', '/', '\\', 'º', '−', '^', 'ʻ', 'ˆ"]

chars_to_ignore_regex = f"[{re.escape(''.join(CHARS_TO_IGNORE))}]"

counter = 0
wer_counter = 0
cer_counter = 0

def main():
    model = AutoModelForCTC.from_pretrained("flozi00/wav2vec2-large-xlsr-53-german-with-lm")
    processor = AutoProcessor.from_pretrained("flozi00/wav2vec2-large-xlsr-53-german-with-lm")

    wer = load_metric("wer")
    cer = load_metric("cer")

    ds = load_dataset("common_voice", "de", split="test")
    #ds = ds.select(range(100))

    def calculate_metrics(batch):
        global counter, wer_counter, cer_counter
        resampled_audio = F.resample(torch.tensor(batch["audio"]["array"]), 48_000, 16_000).numpy()

        input_values = processor(resampled_audio, return_tensors="pt", sampling_rate=16_000).input_values

        with torch.no_grad():
            logits = model(input_values).logits.numpy()[0]


        decoded = processor.decode(logits)
        pred = decoded.text

        ref = re.sub(chars_to_ignore_regex, "", batch["sentence"]).upper()

        wer_result = wer.compute(predictions=[pred], references=[ref])
        cer_result = cer.compute(predictions=[pred], references=[ref])

        counter += 1
        wer_counter += wer_result
        cer_counter += cer_result

        print(f"WER: {(wer_counter/counter)*100} | CER: {(cer_counter/counter)*100}")

        return batch


    ds.map(calculate_metrics, remove_columns=ds.column_names)
    
main()

📚 詳細文檔

測試結果

評估

該模型可按上述 Python 代碼在 Common Voice 的德語測試數據上進行評估。

致謝

聲學模型是 jonatasgrosman 的模型 的副本，用於訓練匹配的 kenlm 語言模型。

📄 許可證

本項目採用 Apache-2.0 許可證。

🔧 技術細節

暫未提供相關技術細節。