Afro-XLMR-large-76L Open-Source Language Model - Supports Text Processing in 76 African Languages

Afro Xlmr Large 76L

Developed by Davlan

AfroXLMR-large-76L is created through MLM (Masked Language Modeling) adaptation of the extended XLM-R-large model, covering 76 widely used languages in Africa, including 4 high-resource languages.

Large Language Model Supports Multiple LanguagesOpen Source License:MIT #African Multilingual Model #76 Language Support #MLM Adaptation

Downloads 921

Release Time : 2/18/2024

Model Overview

This model is a multilingual language model focused on supporting various African languages, suitable for natural language processing tasks such as text classification, translation, and language understanding.

Model Features

Multilingual Support

Supports 76 widely used languages in Africa, including 4 high-resource languages.

Rich Pretraining Corpus

Combines mC4, Wikipedia, and OPUS data to ensure performance in multilingual tasks.

Based on XLM-R-large Architecture

Adapted through MLM from the extended XLM-R-large model, inheriting its robust language understanding capabilities.

Model Capabilities

Text Classification

Language Understanding

Translation

Use Cases

Natural Language Processing

Multilingual Text Classification

Used for classifying texts in various African languages, suitable for tasks like news categorization and sentiment analysis.

Language Translation

Supports translation among multiple African languages, helping to break language barriers.

🚀 afro-xlmr-large-76L

AfroXLMR-large-76L is an MLM adaptation of the expanded XLM-R-large model. It is trained on 76 languages widely spoken in Africa, including 4 high-resource languages, aiming to provide better language processing capabilities for African languages.

✨ Features

Multilingual Support: Covers 76 languages spoken in Africa, enabling cross - language tasks in a wide range of African language scenarios.
Rich Pre - training Corpus: Utilizes a mix of mC4, Wikipedia, and OPUS data for pre - training, enhancing the model's generalization ability.

📚 Documentation

Pre - training corpus

A mix of mC4, Wikipedia and OPUS data

Languages

There are 76 languages available:

English (eng)
Amharic (amh)
Arabic (ara)
Somali (som)
Kiswahili (swa)
Portuguese (por)
Afrikaans (afr)
French (fra)
isiZulu (zul)
Malagasy (mlg)
Hausa (hau)
chiShona (sna)
Egyptian Arabic (arz)
Chichewa (nya)
Igbo (ibo)
isiXhosa (xho)
Yorùbá (yor)
Sesotho (sot)
Kinyarwanda (kin)
Tigrinya (tir)
Tsonga (tso)
Oromo (orm)
Rundi (run)
Northern Sotho (nso)
Ewe (ewe)
Lingala (lin)
Twi (twi)
Nigerian Pidgin (pcm)
Ga (gaa)
Lozi (loz)
Luganda (lug)
Gun (guw)
Bemba (bem)
Efik (efi)
Luvale (lue)
Luba - Lulua (lua)
Tonga (toi)
Tshivenḓa (ven)
Tumbuka (tum)
Tetela (tll)
Isoko (iso)
Kaonde (kqn)
Zande (zne)
Umbundu (umb)
Mossi (mos)
Tiv (tiv)
Luba - Katanga (lub)
Fula (fuv)
San Salvador Kongo (kwy)
Baoulé (bci)
Ruund (rnd)
Luo (luo)
Wolaitta (wal)
Swazi (ssw)
Lunda (lun)
Wolof (wol)
Nyaneka (nyk)
Kwanyama (kua)
Kikuyu (kik)
Fon (fon)
Bambara (bam)
Chokwe (cjk)
Dinka (dik)
Dyula (dyu)
Kabyle (kab)
Kamba (kam)
Kabiyè (kbp)
Kanuri (knc)
Kimbundu (kmb)
Kikongo (kon)
Nuer (nus)
Sango (sag)
Tamasheq (taq)
Tamazight (tzm)
N'ko (nqo)

Acknowledgment

We would like to thank Google Cloud for providing us access to TPU v3 - 8 through the free cloud credits. Model trained using flax, before converted to pytorch.

BibTeX entry and citation info.

@misc{adelani2023sib200,
      title={SIB - 200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects}, 
      author={David Ifeoluwa Adelani and Hannah Liu and Xiaoyu Shen and Nikita Vassilyev and Jesujoba O. Alabi and Yanke Mao and Haonan Gao and Annie En - Shiun Lee},
      year={2023},
      eprint={2309.07445},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

📄 License

This project is under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご