Japanese - Hubert - base Open - source Japanese Speech Model - Supports Japanese speech processing with a large amount of corpus

Japanese Hubert Base

Developed by rinna

Japanese HuBERT base model trained by rinna Co., Ltd., based on approximately 19,000 hours of Japanese speech corpus ReazonSpeech v1.

Speech Recognition

Transformers

JapaneseOpen Source License:Apache-2.0 #Japanese speech feature extraction #Self-supervised learning #Large-scale speech pre-training

Downloads 4,550

Release Time : 4/28/2023

Model Overview

This is a Japanese speech representation learning model based on the HuBERT architecture, primarily used for speech feature extraction and speech-related tasks.

Model Features

Japanese speech optimization

Specially trained and optimized for Japanese speech data

Large-scale training data

Trained using approximately 19,000 hours of Japanese speech corpus ReazonSpeech v1

HuBERT architecture

Adopts HuBERT's self-supervised learning architecture for speech representation learning through hidden unit masking prediction

Model Capabilities

Speech feature extraction

Speech representation learning

Use Cases

Speech processing

Speech feature extraction

Extract high-level feature representations from Japanese speech

Speech-related downstream tasks

Can be used as a pre-trained model for tasks such as speech recognition and speech classification

🚀 `rinna/japanese-hubert-base`

This is a Japanese HuBERT Base model trained by rinna Co., Ltd., which can be used for speech-related tasks.

🚀 Quick Start

This is a Japanese HuBERT Base model trained by rinna Co., Ltd.. You can use it for speech-related tasks.

✨ Features

Model Summary

The model architecture is the same as the original HuBERT Base model, which contains 12 transformer layers with 12 attention heads. The model was trained using code from the official repository, and the detailed training configuration can be found in the same repository and the original paper.

Training

The model was trained on approximately 19,000 hours of the following Japanese speech corpus ReazonSpeech v1.

ReazonSpeech

Contributors

Release Date

April 28, 2023

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

import soundfile as sf
from transformers import AutoFeatureExtractor, AutoModel

model_name = "rinna/japanese-hubert-base"
feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
model.eval()

raw_speech_16kHz, sr = sf.read(audio_file)
inputs = feature_extractor(
    raw_speech_16kHz,
    return_tensors="pt",
    sampling_rate=sr,
)
outputs = model(**inputs)

print(f"Input:  {inputs.input_values.size()}")  # [1, #samples]
print(f"Output: {outputs.last_hidden_state.size()}")  # [1, #frames, 768]

A fairseq checkpoint file can also be available here.

📚 Documentation

How to Cite

@misc{rinna-japanese-hubert-base,
    title = {rinna/japanese-hubert-base},
    author = {Hono, Yukiya and Mitsui, Kentaro and Sawada, Kei},
    url = {https://huggingface.co/rinna/japanese-hubert-base}
}

@inproceedings{sawada2024release,
    title = {Release of Pre-Trained Models for the {J}apanese Language},
    author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh},
    booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
    month = {5},
    year = {2024},
    pages = {13898--13905},
    url = {https://aclanthology.org/2024.lrec-main.1213},
    note = {\url{https://arxiv.org/abs/2404.01657}}
}

References

@article{hsu2021hubert,
    author = {Hsu, Wei-Ning and Bolte, Benjamin and Tsai, Yao-Hung Hubert and Lakhotia, Kushal and Salakhutdinov, Ruslan and Mohamed, Abdelrahman},
    journal = {IEEE/ACM Transactions on Audio, Speech, and Language Processing},
    title = {HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units},
    year = {2021},
    volume = {29},
    pages = {3451-3460},
    doi = {10.1109/TASLP.2021.3122291}
}

📄 License

The Apache 2.0 license

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご