SSA HuBERT Base 60k

🚀 SSA-HuBERT-base-60k: Self-Supervised Speech Model

This self-supervised speech model (SSA-HuBERT-base-60k) is based on the HuBERT Base architecture (~95M params). It addresses the challenge of multilingual speech processing in Sub - Saharan Africa by leveraging nearly 60 000 hours of speech segments, covering 21 languages and variants.

🚀 Quick Start

The model is ready for use after fine - tuning. For ASR fine - tuning, the SpeechBrain toolkit is used, and the FLEURS dataset is applied for each language.

✨ Features

• Multilingual Coverage: Covers 21 languages and variants spoken in Sub - Saharan Africa.
• Large - scale Training: Trained on nearly 60 000 hours of speech segments.
• Self - supervised Learning: Based on the HuBERT Base architecture for effective speech representation learning.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

No code examples are provided in the original document.

📚 Documentation

Model description

This self - supervised speech model (a.k.a. SSA - HuBERT - base - 60k) is based on a HuBERT Base architecture (~95M params) [1]. It was trained on nearly 60 000 hours of speech segments and covers 21 languages and variants spoken in Sub - Saharan Africa.

Pretraining data

• Dataset: The training dataset was composed of both studio recordings (controlled environment, prepared talks) and street interviews (noisy environment, spontaneous speech).
• Languages: Bambara (bam), Dyula (dyu), French (fra), Fula (ful), Fulfulde (ffm), Fulfulde (fuh), Gulmancema (gux), Hausa (hau), Kinyarwanda (kin), Kituba (ktu), Lingala (lin), Luba - Lulua (lua), Mossi (mos), Maninkakan (mwk), Sango (sag), Songhai (son), Swahili (swc), Swahili (swh), Tamasheq (taq), Wolof (wol), Zarma (dje).

ASR fine - tuning

The SpeechBrain toolkit (Ravanelli et al., 2021) is used to fine - tune the model. Fine - tuning is done for each language using the FLEURS dataset [2]. The pretrained model (SSA - HuBERT - base - 60k) is considered as a speech encoder and is fully fine - tuned with two 1024 linear layers and a softmax output at the top.

Results

The following results are obtained in a greedy mode (no language model rescoring). Character error rates (CERs) and Word error rates (WERs) are given in the table below, on the 20 languages of the SSA subpart of the FLEURS dataset.

Reproductibilty

We propose a notebook to reproduce the ASR experiments mentioned in our paper. See SB_ASR_FLEURS_finetuning.ipynb. By using the ASR_FLEURS - swahili_hf.yaml config file, you will be able to run the recipe on Swahili.

🔧 Technical Details

The model is based on the HuBERT Base architecture. The pretraining data consists of studio recordings and street interviews. For fine - tuning, the SpeechBrain toolkit is used with the FLEURS dataset, and the model is fully fine - tuned with two 1024 linear layers and a softmax output at the top.

📄 License

This model is released under the CC - by - NC 4.0 conditions.

Publication

This model were presented at AfricaNLP 2024. The associated paper is available here: Africa - Centric Self - Supervised Pre - Training for Multilingual Speech Representation in a Sub - Saharan Context

Citation

Please cite our paper when using SSA - HuBERT - base - 60k model:

Caubrière, A., & Gauthier, E. (2024). Africa - Centric Self - Supervised Pre - Training for Multilingual Speech Representation in a Sub - Saharan Context. In 5th Workshop on African Natural Language Processing (AfricaNLP 2024).

Bibtex citation:

@inproceedings{caubri{\`e}re2024ssaspeechssl,    
    title={Africa - Centric Self - Supervised Pretraining for Multilingual Speech Representation in a Sub - Saharan Context},    
    author={Antoine Caubri{\`e}re and Elodie Gauthier},    
    booktitle={5th Workshop on African Natural Language Processing},    
    year={2024},    
    url={https://openreview.net/forum?id=zLOhcft2E7}}

References

[1] Wei - Ning Hsu, Benjamin Bolte, Yao - Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. HuBERT: Self - Supervised Speech Representation Learning by Masked Prediction of Hidden Units. In 2021 IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp.3451–3460, 2021. doi: 10.1109/TASLP.2021.3122291. [2] Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. Fleurs: Few - shot learning evaluation of universal representations of speech. In 2022 IEEE Spoken Language Technology Workshop (SLT), pp. 798–805, 2022. doi: 10.1109/SLT54892.2023.10023141.