Multilingual Distilwhisper 28k

🚀 Multilingual Distilwhisper

Multilingual Distilwhisper enhances ASR performance in target languages by integrating lightweight CLSR modules on whisper - small, trained with a combination of cross - entropy and knowledge distillation losses.

🚀 Quick Start

The code for training and inference can be found at: GitHub Repository

✨ Features

Multilingual Distilwhisper adds lightweight CLSR modules on top of whisper - small to improve ASR performance in target languages. These modules are trained using a combination of cross - entropy (ASR) and knowledge distillation losses, with whisper - large - v2 as the teacher model. For more details, refer to the ICASSP 2024 paper: arxiv.org/abs/2311.01070

📄 License

This project is licensed under the MIT license.

📚 Documentation

Citation

@inproceedings{ferraz2024distilwhisper,
  title={Multilingual DistilWhisper: Efficient Distillation of Multi - task Speech Models via Language - Specific Experts},
  author={Ferraz, Thomas Palmeira and Boito, Marcely Zanon and Brun, Caroline and Nikoulina, Vassilina},
  booktitle={ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year={2024},
  organization={IEEE}
}