Belle-whisper-large-v3-zh Open Source Chinese Speech Recognition Model - Improved Performance and Precise Recognition of Chinese Speech

Belle Whisper Large V3 Zh

Developed by BELLE-2

A Chinese speech recognition model fine-tuned and optimized based on whisper-large-v3, showing significant performance improvements in multiple Chinese speech benchmarks

Speech Recognition

Transformers

Open Source License:Apache-2.0 #Chinese Speech Recognition #High Accuracy Transcription #Meeting Scenario Optimization

Downloads 1,666

Release Time : 3/11/2024

Model Overview

This model is a Whisper model optimized for Chinese speech recognition tasks, achieving performance enhancements through full-parameter fine-tuning on multiple Chinese speech datasets

Model Features

Chinese Speech Recognition Optimization

Specifically fine-tuned for Chinese speech, achieving performance improvements of 24%-65% in multiple Chinese speech benchmarks

Complex Scenario Adaptation

Excellent performance in complex acoustic scenarios such as meetings, showing significant improvements over previous models

Multi-Dataset Training

Trained using multiple high-quality Chinese speech datasets including AISHELL-1, AISHELL-2, WenetSpeech, and HKUST

Model Capabilities

Chinese Speech-to-Text

High Accuracy Speech Recognition

Complex Scenario Speech Processing

Use Cases

Speech Transcription

Meeting Minutes

Convert meeting recordings into text transcripts

Achieves an error rate of only 11.246% on the wenetspeech_meeting dataset

Voice Input

Provide Chinese voice input functionality for applications

🚀 Belle-whisper-large-v3-zh

Fine-tune the whisper-large-v3 model to enhance Chinese speech recognition capabilities, achieving a 24 - 65% relative performance improvement on Chinese ASR benchmarks.

🚀 Quick Start

If you find this model helpful, please like this model and star us on GitHub and GitHub.

✨ Features

Belle-whisper-large-v3-zh fine-tunes the whisper-large-v3 model to enhance Chinese speech recognition capabilities. It demonstrates a 24 - 65% relative improvement in performance on Chinese ASR benchmarks, including AISHELL1, AISHELL2, WENETSPEECH, and HKUST.

💻 Usage Examples

Basic Usage

from transformers import pipeline

transcriber = pipeline(
  "automatic-speech-recognition", 
  model="BELLE-2/Belle-whisper-large-v3-zh"
)

transcriber.model.config.forced_decoder_ids = (
  transcriber.tokenizer.get_decoder_prompt_ids(
    language="zh", 
    task="transcribe"
  )
)

transcription = transcriber("my_audio.wav")

📚 Documentation

Fine-tuning

Property	Details
Model	Belle-whisper-large-v3-zh
(Re)Sample Rate	16KHz
Train Datasets	AISHELL-1 AISHELL-2 WenetSpeech HKUST
Fine-tuning (full or peft)	full fine-tuning

If you want to fine-tune the model on your datasets, please refer to the GitHub repo.

CER(%) ↓

Model	Language Tag	aishell_1_test(↓)	aishell_2_test(↓)	wenetspeech_net(↓)	wenetspeech_meeting(↓)	HKUST_dev(↓)
whisper-large-v3	Chinese	8.085	5.475	11.72	20.15	28.597
Belle-whisper-large-v2-zh	Chinese	2.549	3.746	8.503	14.598	16.289
Belle-whisper-large-v3-zh	Chinese	2.781	3.786	8.865	11.246	16.440

It is worth mentioning that compared to Belle-whisper-large-v2-zh, Belle-whisper-large-v3-zh has a significant improvement in complex acoustic scenes (such as wenetspeech_meeting).

📄 License

This project is licensed under the Apache-2.0 license.

📖 Citation

Please cite our paper and GitHub when using our code, data, or model.

@misc{BELLE,
  author = {BELLEGroup},
  title = {BELLE: Be Everyone's Large Language model Engine},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/LianjiaTech/BELLE}},
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご