Open-source model whisper-large-v3-japanese-4k-steps-ct2 - Supports multilingual speech recognition, with better Japanese recognition results

Whisper Large V3 Japanese 4k Steps Ct2

Developed by JhonVanced

This is a CTranslate2 converted version of the OpenAI Whisper large-v3 model, specifically fine-tuned for Japanese with an additional 4,000 training steps, supporting multilingual speech recognition.

Speech Recognition Supports Multiple LanguagesOpen Source License:MIT #Multilingual speech recognition #Japanese optimization #Efficient inference

Downloads 54

Release Time : 2/20/2024

Model Overview

A speech recognition model based on Whisper large-v3, converted to CTranslate2 format for improved inference efficiency, supporting multilingual speech-to-text tasks.

Model Features

Efficient inference

After conversion to CTranslate2 format, inference speed is faster than the original PyTorch implementation

Multilingual support

Supports speech recognition for over 100 languages

Japanese optimization

Specifically fine-tuned for Japanese with an additional 4,000 training steps

FP16 quantization

Model weights are saved in FP16 format, allowing adjustment of computation precision during loading

Model Capabilities

Speech-to-text

Multilingual speech recognition

Audio transcription

Use Cases

Media transcription

Podcast transcription

Automatically transcribe podcast audio content into text

High-accuracy transcription results with multilingual support

Video subtitle generation

Automatically generate subtitles for video content

Supports subtitle generation in multiple languages

Meeting minutes

Meeting recording transcription

Automatically convert meeting recordings into text records

Improves meeting documentation efficiency and facilitates subsequent retrieval

Language learning

Language learning assistance

Help language learners practice listening and pronunciation

Provides accurate speech recognition feedback

🚀 Whisper large-v3 model for CTranslate2

This repository offers the conversion of the drewschaub/whisper-large-v3-japanese-4k-steps model into the CTranslate2 model format. It enables utilization in CTranslate2 and related projects like faster-whisper.

🚀 Quick Start

This model can be used in CTranslate2 or projects based on CTranslate2.

✨ Features

Supports multiple languages including English, Chinese, German, Spanish, Russian, Korean, French, Japanese, Portuguese, Turkish, Polish, Catalan, Dutch, Arabic, Swedish, Italian, Indonesian, Hindi, Finnish, Vietnamese, Hebrew, Ukrainian, Greek, Malay, Czech, Romanian, Danish, Hungarian, Tamil, Norwegian, Thai, Urdu, Croatian, Bulgarian, Lithuanian, Latin, Maori, Malayalam, Welsh, Slovak, Telugu, Persian, Latvian, Bengali, Serbian, Azerbaijani, Slovenian, Kannada, Estonian, Macedonian, Breton, Basque, Icelandic, Armenian, Nepali, Mongolian, Bosnian, Kazakh, Albanian, Swahili, Galician, Marathi, Punjabi, Sinhala, Khmer, Shona, Yoruba, Somali, Afrikaans, Occitan, Georgian, Belarusian, Tajik, Sindhi, Gujarati, Amharic, Yiddish, Lao, Uzbek, Faroese, Haitian Creole, Pashto, Turkmen, Norwegian Nynorsk, Maltese, Sanskrit, Luxembourgish, Burmese, Tibetan, Tagalog, Malagasy, Assamese, Tatar, Hawaiian, Lingala, Hausa, Bashkir, Javanese, Sundanese, and Cantonese.
Applicable for audio and automatic speech recognition tasks.

📦 Installation

No specific installation steps are provided in the original README.

💻 Usage Examples

Basic Usage

from faster_whisper import WhisperModel

model = WhisperModel("large-v3")

segments, info = model.transcribe("audio.mp3")
for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

🔧 Technical Details

Conversion details

The original model was converted with the following command:

ct2-transformers-converter --model drewschaub/whisper-large-v3-japanese-4k-steps --output_dir whisper-large-v3-japanese-4k-steps-ct2 \
    --copy_files tokenizer.json preprocessor_config.json --quantization float16

Note that the model weights are saved in FP16. This type can be changed when the model is loaded using the compute_type option in CTranslate2.

📚 Documentation

For more information about the original model, see its model card.

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご