Mms Zeroshot 300m

🚀 Massively Multilingual Speech (MMS) - Finetuned ASR - ALL

This model from the MMS Zero-shot project can transcribe speech in almost any language with just a small amount of unlabeled text in the target language.

🚀 Quick Start

This is a checkpoint of the MMS Zero-shot project. It's a model designed to transcribe the speech of nearly any language, requiring only a small quantity of unlabeled text in the new language. The approach relies on a multilingual acoustic model trained on data from 1,150 languages (utilizing the data of MMS). This model outputs transcriptions in an intermediate representation (uroman tokens). A small amount of text in the new, unseen language is then mapped to this intermediate representation. At inference time, this mapping, along with an optional language model, enables the transcription of a new language.

✨ Features

• Capable of transcribing speech in almost any language with minimal unlabeled text.
• Utilizes a multilingual acoustic model trained on 1,150 languages.
• Outputs transcriptions in an intermediate representation for broader language support.

💻 Usage Examples

Please have a look at the official space for an example on using the model.

📚 Documentation

Model details

@article{zhao2024scaling,
  title={Scaling A Simple Approach to Zero-Shot Speech Recognition},
  author={Zhao, Jinming and Pratap, Vineel and Auli, Michael},
  journal={arXiv preprint arXiv:2407.17852},
  year={2024}
}

Additional Links

• Paper
• GitHub Repository
• Official Space

📄 License

This model is released under the CC-BY-NC 4.0 license.