Teochew Whisper Medium

🚀 Teochew Whisper Medium

This is a fine - tuned version of the Whisper medium model, designed to recognize the Teochew language (潮州话), a dialect in the Min Nan family spoken in southern China.

🚀 Quick Start

For a detailed documentation of how this model was trained, please refer to this video: Training Documentation Video

✨ Features

• Language Recognition: Specifically fine - tuned to recognize the Teochew language.
• Diverse Data Training: Trained on a variety of audio sources including movies, TV shows, and comedies.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

The following script downloads the model and starts a demo using Gradio to run the model:

import torch
import torchaudio
from transformers import WhisperProcessor, WhisperForConditionalGeneration
import gradio as gr

DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
WHISPER_SAMPLE_RATE = 16000

processor = WhisperProcessor.from_pretrained("openai/whisper-medium")
model = WhisperForConditionalGeneration.from_pretrained(
    "efficient-nlp/teochew-whisper-medium"
).to(DEVICE)


def preprocess_audio(audio_path: str) -> torch.Tensor:
    audio, sample_rate = torchaudio.load(audio_path)
    # Resample if necessary
    if sample_rate != WHISPER_SAMPLE_RATE:
        resampler = torchaudio.transforms.Resample(
            orig_freq=sample_rate, new_freq=WHISPER_SAMPLE_RATE
        )
        audio = resampler(audio)
    # Convert to mono
    if audio.shape[0] > 1:
        audio = torch.mean(audio, dim=0)
    return audio.squeeze()


def transcribe(audio_path: str) -> str:
    audio_input = preprocess_audio(audio_path)
    input_features = processor(
        audio_input,
        sampling_rate=WHISPER_SAMPLE_RATE,
        return_tensors="pt",
        language="Chinese",
    ).input_features.to(DEVICE)

    predicted_ids = model.generate(input_features)
    transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
    return transcription


iface = gr.Interface(
    fn=transcribe,
    inputs=gr.Audio(type="filepath"),
    outputs="text",
    title="Teochew Speech Recognition",
)
iface.launch()

📚 Documentation

Training Data

The model was fine - tuned on approximately 35 hours of audio data derived from Teochew language movies, TV shows, and comedies.

Evaluation Metrics

On our private test set, we obtained the following Word Error Rate (WER) metrics:

Known Limitations

⚠️ Important Note

This model has been trained on short audio clips and may struggle with audio that is longer than 10 seconds.

📄 License

This project is licensed under the MIT license.