kotoba-whisper-v2.0-faster Open-source Speech Recognition Model - Free Deployment, Efficiently Convert Japanese Speech to Text

Kotoba Whisper V2.0 Faster

Developed by kotoba-tech

A Whisper speech recognition model optimized for CTranslate2, specifically tailored for Japanese, providing efficient speech-to-text functionality.

Speech Recognition JapaneseOpen Source License:MIT #Japanese Speech Recognition #Efficient Inference #Long Audio Processing

Downloads 202

Release Time : 9/17/2024

Model Overview

This model is the CTranslate2 converted version of kotoba-whisper-v2.0, supporting Japanese speech recognition and suitable for applications requiring fast and efficient speech-to-text conversion.

Model Features

Efficient Inference

Optimized with CTranslate2, offering faster inference speeds compared to the original Whisper model.

Japanese Optimization

Specially optimized for Japanese speech recognition, delivering more accurate transcription results.

FP16 Quantization

Model weights are saved in FP16 format, supporting flexible computation type selection.

Model Capabilities

Japanese Speech Recognition

Long-format Audio Processing

Real-time Transcription

Use Cases

Speech-to-Text

Japanese Meeting Minutes

Automatically transcribe Japanese meeting recordings into text for easy organization and reference.

Efficient and accurate transcription results, supporting long-duration audio processing.

Japanese Podcast Transcription

Transcribe Japanese podcast content into text for easy indexing and searching.

Supports chunk processing, improving the accuracy of long audio transcription.

🚀 Whisper kotoba-whisper-v2.0 model for CTranslate2

This repository contains the conversion of kotoba-tech/kotoba-whisper-v2.0 to the CTranslate2 model format. This model can be used in CTranslate2 or projects based on CTranslate2 such as faster-whisper.

🚀 Quick Start

Installation

Install the library and download the sample audio.

pip install faster-whisper
wget https://huggingface.co/kotoba-tech/kotoba-whisper-v1.0-ggml/resolve/main/sample_ja_speech.wav

Inference

Perform inference with the kotoba-whisper-v2.0-faster.

from faster_whisper import WhisperModel

model = WhisperModel("kotoba-tech/kotoba-whisper-v2.0-faster")

segments, info = model.transcribe("sample_ja_speech.wav", language="ja", chunk_length=15, condition_on_previous_text=False)
for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

💻 Usage Examples

Basic Usage

The above example shows the basic usage of the model for audio transcription.

📊 Benchmark

We measure the inference speed of different kotoba-whisper-v2.0 implementations with four different Japanese speech audio on a MacBook Pro with the following spec:

Apple M2 Pro
32GB
14-inch, 2023
OS Sonoma Version 14.4.1 (23E224)

audio file	audio duration (min)	whisper.cpp (sec)	faster-whisper (sec)	hf pipeline (sec)
audio 1	50.3	581	2601	807
audio 2	5.6	41	73	61
audio 3	4.9	30	141	54
audio 4	5.6	35	126	69

Scripts to re-run the experiment can be found below:

Also, currently whisper.cpp and faster-whisper support the sequential long-form decoding, and only the Huggingface pipeline supports the chunked long-form decoding, which we empirically found better than the sequential long-form decoding.

🔧 Technical Details

Conversion details

The original model was converted with the following command:

ct2-transformers-converter --model kotoba-tech/kotoba-whisper-v2.0 --output_dir kotoba-whisper-v2.0-faster \
    --copy_files tokenizer.json preprocessor_config.json --quantization float16

Note that the model weights are saved in FP16. This type can be changed when the model is loaded using the compute_type option in CTranslate2.

📚 Documentation

For more information about the kotoba-whisper-v2.0, refer to the original model card.

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご