Darijabert Arabizi

🚀 DarijaBERT - Arabizi Version

AIOX Lab and SI2M Lab INSEA present the first intelligent Open - Source system for understanding Moroccan dialectal language "Darija" in Arabizi, offering a valuable resource for NLP research.

🚀 Quick Start

AIOX Lab and SI2M Lab INSEA have collaborated to provide researchers, industrialists, and the NLP (Natural Language Processing) community with the first intelligent Open Source system capable of understanding the Moroccan dialectal language "Darija".

DarijaBERT is the first BERT model tailored for the Moroccan Arabic dialect known as “Darija”. It shares the same architecture as BERT - base, but without the Next Sentence Prediction (NSP) objective. This particular model is the Arabizi - specific version of DarijaBERT, trained on approximately 4.6 million sequences of the Darija dialect written in Latin letters.

✨ Features

• Dialect - Specific: Designed specifically for the Moroccan Arabic dialect "Darija".
• Arabizi Compatibility: Trained on Darija text written in Latin letters (Arabizi).
• Open - Source: An open - source system, accessible to the NLP community.

📦 Installation

The model can be loaded directly using the Huggingface library:

from transformers import AutoTokenizer, AutoModel
DarijaBERT_tokenizer = AutoTokenizer.from_pretrained("SI2M-Lab/DarijaBERT-arabizi")
DarijaBert_model = AutoModel.from_pretrained("SI2M-Lab/DarijaBERT-arabizi")

📚 Documentation

Model Details

The model was trained on a dataset sourced from Youtube comments. More details about DarijaBert are available in the dedicated GitHub repository.

Information Table

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModel
DarijaBERT_tokenizer = AutoTokenizer.from_pretrained("SI2M-Lab/DarijaBERT-arabizi")
DarijaBert_model = AutoModel.from_pretrained("SI2M-Lab/DarijaBERT-arabizi")

📄 License

Citation

If you use our models for your scientific publication, or if you find the resources in this repository useful, please cite our paper as follows (to be updated):

@article{gaanoun2023darijabert,
  title={Darijabert: a Step Forward in Nlp for the Written Moroccan Dialect},
  author={Gaanoun, Kamel and Naira, Abdou Mohamed and Allak, Anass and Benelallam, Imade},
  year={2023}
}

Acknowledgments

We gratefully acknowledge Google’s TensorFlow Research Cloud (TRC) program for providing us with free Cloud TPUs.

Warning

⚠️ Important Note

This model being trained on texts from social networks, it can unfortunately generate toxic outputs reflecting part of the learned data.