Lite Whisper Large V3 Turbo Acc

🚀 Model Card for Lite-Whisper large-v3-turbo-acc

Lite-Whisper is a compressed version of OpenAI Whisper with LiteASR. It aims to optimize the performance of speech recognition while reducing the model size. For more details, check out our GitHub repository and paper.

✨ Features

• Compression: It's a compressed version of OpenAI Whisper, which helps in reducing the model size.
• Performance: Maintains competitive performance in automatic speech recognition tasks.

📦 Installation

The model can be easily used with the HuggingFace Transformers library. You can load the model weights from here.

💻 Usage Examples

Basic Usage

import librosa 
import torch
from transformers import AutoProcessor, AutoModel

device = "cuda:0"
dtype = torch.float16

# load the compressed Whisper model
model = AutoModel.from_pretrained(
    "efficient-speech/lite-whisper-large-v3-turbo", 
    trust_remote_code=True, 
)
model.to(dtype).to(device)

# we use the same processor as the original model
processor = AutoProcessor.from_pretrained("openai/whisper-large-v3")

# set the path to your audio file
path = "path/to/audio.wav"
audio, _ = librosa.load(path, sr=16000)

input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features
input_features = input_features.to(dtype).to(device)

predicted_ids = model.generate(input_features)
transcription = processor.batch_decode(
    predicted_ids, 
    skip_special_tokens=True
)[0]

print(transcription)

📚 Documentation

Benchmark Results

The following table shows the average word error rate (WER) evaluated on the ESB datasets:

Model Information

📄 License

This project is licensed under the apache-2.0 license.

📖 Citation

If you use LiteASR in your research, please cite the following paper:

@misc{kamahori2025liteasrefficientautomaticspeech,
      title={LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation}, 
      author={Keisuke Kamahori and Jungo Kasai and Noriyuki Kojima and Baris Kasikci},
      year={2025},
      eprint={2502.20583},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2502.20583}, 
}