Whisper Large V3 Turbo Swissgerman

🚀 Whisper Large V3 Turbo (Swiss German Fine-Tuned with QLoRa)

This repository offers a fine - tuned OpenAI's Whisper Large V3 Turbo model for Swiss German dialects using QLoRa, achieving top - notch performance in Swiss German automatic speech recognition.

🚀 Quick Start

This model can be used directly with the Hugging Face Transformers library for tasks requiring Swiss German ASR.

✨ Features

This repository contains a fine - tuned version of OpenAI's Whisper Large V3 Turbo model, adapted specifically for Swiss German dialects using QLoRa optimization. The model achieves state - of - the - art performance for Swiss German automatic speech recognition (ASR).

📚 Documentation

Model Summary

Performance Metrics

• Word Error Rate (WER): 17.5%
• BLEU Score: 65.0

The model's performance has been evaluated across multiple datasets representing diverse dialectal and demographic distributions in Swiss German.

Dataset Summary

The model has been trained and evaluated on a comprehensive suite of Swiss German datasets:

1. SDS - 200 Corpus
   • Size: 200 hours
   • Description: A corpus covering all Swiss German dialects.
2. STT4SG - 350
   • Size: 343 hours
   • Description: Balanced distribution across Swiss German dialects and demographics, including gender representation.
   • Dataset Link
3. SwissDial - Zh v1.1
   • Size: 24 hours
   • Description: A dataset with balanced representation of Swiss German dialects.
   • Dataset Link
4. Swiss Parliament Corpus V2 (SPC)
   • Size: 293 hours
   • Description: Parliament recordings across Swiss German dialects.
   • Dataset Link
5. ASGDTS (All Swiss German Dialects Test Set)
   • Size: 13 hours
   • Description: A stratified dataset closely resembling real - world Swiss German dialect distribution.
   • Dataset Link

Results Across Datasets

WER Scores

BLEU Scores

Visual Results

• WER and BLEU Scores Across Datasets
• WER Scores Across Datasets
• BLEU Scores Across Datasets

📄 License

The model uses the MIT license. However, due to some datasets' licenses the model had to be taken down.

🙏 Acknowledgments

Special thanks to the creators and maintainers of the datasets used in this work:

• Swiss NLP
• ETH Zurich
• FHNW

And to the University of Geneva for allowing us access to their High Performance Computing cluster on which the model has been trained.

📑 Citation

If you use this model in your work, please cite this repository as follows:

@misc{whisper-large-v3-turbo-swissgerman,
  author = {Nizar Michaud},
  title = {Whisper Large V3 Turbo Fine-Tuned for Swiss German},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/nizarmichaud/whisper-large-v3-turbo-swissgerman},
  doi = 10.57967/hf/3858,
}