Wav2vec2 Large Xlsr 53 German With Lm

🚀 XLSR Wav2Vec2 德语模型

本项目的 XLSR Wav2Vec2 德语模型，可用于自动语音识别任务，在 Common Voice 德语测试数据上有出色表现。

🚀 快速开始

本模型可在 Common Voice 的德语测试数据上进行评估，具体评估方式见下方代码示例。

✨ 主要特性

• 数据集：使用 Common Voice 数据集进行训练。
• 评估指标：使用字错率（WER）和字符错误率（CER）进行评估。
• 适用领域：适用于音频、自动语音识别等领域。

📦 安装指南

暂未提供相关安装步骤。

💻 使用示例

基础用法

以下代码展示了如何在 Common Voice 的德语测试数据上评估该模型：

import torchaudio.functional as F
import torch
from transformers import AutoModelForCTC, AutoProcessor
import re
from datasets import load_dataset, load_metric

CHARS_TO_IGNORE = [",", "?", "¿", ".", "!", "¡", ";", "；", ":", '""', "%", '"', "�", "ʿ", "·", "჻", "~", "՞",
                   "؟", "،", "।", "॥", "«", "»", "„", "“", "”", "「", "」", "‘", "’", "《", "》", "(", ")", "[", "]",
                   "{", "}", "=", "`", "_", "+", "<", ">", "…", "–", "°", "´", "ʾ", "‹", "›", "©", "®", "—", "→", "。",
                   "、", "﹂", "﹁", "‧", "～", "﹏", "，", "｛", "｝", "（", "）", "［", "］", "【", "】", "‥", "〽",
                   "『', '』', '〝', '〟', '⟨', '⟩', '〜', '：', '！', '？', '♪', '؛', '/', '\\', 'º', '−', '^', 'ʻ', 'ˆ"]

chars_to_ignore_regex = f"[{re.escape(''.join(CHARS_TO_IGNORE))}]"

counter = 0
wer_counter = 0
cer_counter = 0

def main():
    model = AutoModelForCTC.from_pretrained("flozi00/wav2vec2-large-xlsr-53-german-with-lm")
    processor = AutoProcessor.from_pretrained("flozi00/wav2vec2-large-xlsr-53-german-with-lm")

    wer = load_metric("wer")
    cer = load_metric("cer")

    ds = load_dataset("common_voice", "de", split="test")
    #ds = ds.select(range(100))

    def calculate_metrics(batch):
        global counter, wer_counter, cer_counter
        resampled_audio = F.resample(torch.tensor(batch["audio"]["array"]), 48_000, 16_000).numpy()

        input_values = processor(resampled_audio, return_tensors="pt", sampling_rate=16_000).input_values

        with torch.no_grad():
            logits = model(input_values).logits.numpy()[0]


        decoded = processor.decode(logits)
        pred = decoded.text

        ref = re.sub(chars_to_ignore_regex, "", batch["sentence"]).upper()

        wer_result = wer.compute(predictions=[pred], references=[ref])
        cer_result = cer.compute(predictions=[pred], references=[ref])

        counter += 1
        wer_counter += wer_result
        cer_counter += cer_result

        print(f"WER: {(wer_counter/counter)*100} | CER: {(cer_counter/counter)*100}")

        return batch


    ds.map(calculate_metrics, remove_columns=ds.column_names)
    
main()

📚 详细文档

测试结果

评估

该模型可按上述 Python 代码在 Common Voice 的德语测试数据上进行评估。

致谢

声学模型是 jonatasgrosman 的模型 的副本，用于训练匹配的 kenlm 语言模型。

📄 许可证

本项目采用 Apache-2.0 许可证。

🔧 技术细节

暂未提供相关技术细节。