Japanese Wav2vec2 Base

🚀 rinna/japanese-wav2vec2-base

这是由rinna株式会社训练的日语wav2vec 2.0基础模型，可用于语音特征提取。

🚀 快速开始

本模型是日语wav2vec 2.0基础模型，由rinna株式会社训练。以下是使用该模型的示例代码：

import soundfile as sf
from transformers import AutoFeatureExtractor, AutoModel

model_name = "rinna/japanese-wav2vec2-base"
feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
model.eval()

raw_speech_16kHz, sr = sf.read(audio_file)
inputs = feature_extractor(
    raw_speech_16kHz,
    return_tensors="pt",
    sampling_rate=sr,
)
outputs = model(**inputs)

print(f"Input:  {inputs.input_values.size()}")  # [1, #samples]
print(f"Output: {outputs.last_hidden_state.size()}")  # [1, #frames, 768]

另外，fairseq的检查点文件可以在此处获取。

✨ 主要特性

模型概述

该模型的架构与原始wav2vec 2.0基础模型相同，包含12个具有12个注意力头的Transformer层。模型使用官方仓库中的代码进行训练，详细的训练配置可以在该仓库和原始论文中找到。

训练数据

模型在约19000小时的日语语音语料库ReazonSpeech v1上进行训练，该语料库可在ReazonSpeech获取。

贡献者

• 保野悠哉
• 三井健太郎
• 泽田圭

发布日期

2024年3月7日

📚 详细文档

引用方式

@misc{rinna-japanese-wav2vec2-base,
    title = {rinna/japanese-wav2vec2-base},
    author = {Hono, Yukiya and Mitsui, Kentaro and Sawada, Kei},
    url = {https://huggingface.co/rinna/japanese-wav2vec2-base}
}

@inproceedings{sawada2024release,
    title = {Release of Pre-Trained Models for the {J}apanese Language},
    author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh},
    booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
    month = {5},
    year = {2024},
    pages = {13898--13905},
    url = {https://aclanthology.org/2024.lrec-main.1213},
    note = {\url{https://arxiv.org/abs/2404.01657}}
}

参考文献

@inproceedings{baevski2020wav2vec,
    title = {wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations},
    author = {Baevski, Alexei and Zhou, Yuhao and Mohamed, Abdelrahman and Auli, Michael},
    booktitle = {Advances in Neural Information Processing Systems},
    year = {2020},
    volume = {33},
    pages = {12449--12460},
    url = {https://proceedings.neurips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html}
}

📄 许可证

本模型采用Apache 2.0许可证。