Kotoba Whisper V2.0

🚀 Kotoba-Whisper (v2.0)

Kotoba-Whisper is a collection of distilled Whisper models for Japanese Automatic Speech Recognition (ASR). It is developed through the collaboration between Asahi Ushio and Kotoba Technologies. This model is 6.3x faster than Whisper large-v3 while maintaining a low error rate.

faster-whisper weight, whisper.cpp weight, pipeline with stable-ts/punctuation

✨ Features

• Knowledge Distillation: Following the work of distil-whisper (Robust Knowledge Distillation via Large-Scale Pseudo Labelling), it uses OpenAI's Whisper large-v3 as the teacher model. The student model consists of the full encoder of the teacher large-v3 model and a two - layer decoder initialized from the first and last layers of the large-v3 model.
• High Efficiency: Kotoba-Whisper is 6.3x faster than large-v3, making it more suitable for real - time applications.
• Low Error Rate: It achieves competitive CER and WER on both in - domain and out - of - domain test sets compared to other Whisper models.

📚 Documentation

Model Training

As the successor of kotoba-whisper-v1.0, kotoba-whisper-v2.0 is trained on the all subset of ReazonSpeech, which is the largest speech - transcription paired dataset in Japanese extracted from Japanese TV audio recordings. After removing transcriptions with more than 10 WER, it amounts to 7,203,957 audio clips (5 sec audio with 18 text tokens on average). The model was trained for 8 epochs with a batch size of 256 at a sampling rate of 16kHz. The training and evaluation code to reproduce kotoba-whisper is available at https://github.com/kotoba-tech/kotoba-whisper.

Evaluation Results

CER

WER

📄 License

This project is licensed under the Apache-2.0 license.

Additional Information

Tags

• audio
• automatic-speech-recognition
• hf-asr-leaderboard

Pipeline Tag

automatic-speech-recognition

Metrics

• wer
• cer

Datasets

• japanese-asr/whisper_transcriptions.reazonspeech.all
• japanese-asr/whisper_transcriptions.reazonspeech.all.wer_10.0
• japanese-asr/whisper_transcriptions.reazonspeech.all.wer_10.0.vectorized

Widget Examples

• CommonVoice 8.0 (Test Split): Audio Sample
• JSUT Basic 5000: Audio Sample
• ReazonSpeech (Test Split): Audio Sample