Mms 1b L1107

🚀 Massively Multilingual Speech (MMS) - Finetuned ASR - L1107

This checkpoint represents a model fine - tuned for multi - lingual Automatic Speech Recognition (ASR). It's part of Facebook's Massive Multilingual Speech project, capable of transcribing over 1000 languages.

🚀 Quick Start

This MMS checkpoint can be used with Transformers to transcribe audio of 1107 different languages. The following is a step - by - step guide on how to use it:

1. Install Dependencies

First, install transformers and some other necessary libraries:

pip install torch accelerate torchaudio datasets
pip install --upgrade transformers

Note: In order to use MMS you need to have at least transformers >= 4.30 installed. If the 4.30 version is not yet available on PyPI make sure to install transformers from source:

pip install git+https://github.com/huggingface/transformers.git

2. Load Audio Samples

Next, load a couple of audio samples via datasets. Make sure that the audio data is sampled to 16000 kHz.

from datasets import load_dataset, Audio

# English
stream_data = load_dataset("mozilla - foundation/common_voice_13_0", "en", split="test", streaming=True)
stream_data = stream_data.cast_column("audio", Audio(sampling_rate=16000))
en_sample = next(iter(stream_data))["audio"]["array"]

# French
stream_data = load_dataset("mozilla - foundation/common_voice_13_0", "fr", split="test", streaming=True)
stream_data = stream_data.cast_column("audio", Audio(sampling_rate=16000))
fr_sample = next(iter(stream_data))["audio"]["array"]

3. Load the Model and Processor

from transformers import Wav2Vec2ForCTC, AutoProcessor
import torch

model_id = "facebook/mms - 1b - l1107"

processor = AutoProcessor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id)

4. Process Audio and Transcribe

Now process the audio data, pass the processed audio data to the model and transcribe the model output, just like we usually do for Wav2Vec2 models such as [facebook/wav2vec2 - base - 960h](https://huggingface.co/facebook/wav2vec2 - base - 960h)

inputs = processor(en_sample, sampling_rate=16_000, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs).logits

ids = torch.argmax(outputs, dim=-1)[0]
transcription = processor.decode(ids)
# 'joe keton disapproved of films and buster also had reservations about the media'

5. Switch Languages

We can keep the same model in memory and simply switch out the language adapters by calling the convenient load_adapter() function for the model and set_target_lang() for the tokenizer. We pass the target language as an input - "fra" for French.

processor.tokenizer.set_target_lang("fra")
model.load_adapter("fra")

inputs = processor(fr_sample, sampling_rate=16_000, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs).logits

ids = torch.argmax(outputs, dim=-1)[0]
transcription = processor.decode(ids)
# "ce dernier est volé tout au long de l'histoire romaine"

In the same way the language can be switched out for all other supported languages. Please have a look at:

processor.tokenizer.vocab.keys()

For more details, please have a look at the official docs.

✨ Features

• Multilingual Support: This model supports 1107 languages, providing a wide - range of language coverage for ASR tasks.
• Adapter - Based: It makes use of adapter models to handle different languages efficiently.
• Fine - Tuned: Finetuned for better performance on multi - lingual ASR tasks.

📦 Installation

The installation steps are included in the Quick Start section. You need to install the necessary libraries via pip and ensure that transformers is at least version 4.30.

📚 Documentation

Supported Languages

This model supports 1107 languages. Unclick the following to toggle all supported languages of this checkpoint in [ISO 639 - 3 code](https://en.wikipedia.org/wiki/ISO_639 - 3). You can find more details about the languages and their ISO 649 - 3 codes in the MMS Language Coverage Overview.

Model Details

This checkpoint is based on the Wav2Vec2 architecture and makes use of adapter models to transcribe 1000+ languages. The checkpoint consists of 1 billion parameters and has been fine - tuned from [facebook/mms - 1b](https://huggingface.co/facebook/mms - 1b) on 1107 languages.

Additional Links

• [Massive Multilingual Speech project](https://research.facebook.com/publications/scaling - speech - technology - to - 1000 - languages/)
• Wav2Vec2 architecture
• Official docs

📄 License

This project is licensed under the cc - by - nc - 4.0 license.