Arabic Xlm Xnli

🚀 XLM-ROBERTA-BASE-XNLI-AR

This model is designed for zero-shot text classification in hate speech detection, especially effective in Arabic.

🚀 Quick Start

Usage with Zero-Shot Classification pipeline

from transformers import pipeline
classifier = pipeline("zero-shot-classification",
                      model="morit/arabic_xlm_xnli")

✨ Features

• Based on the XLM-Roberta-base model, pre-trained on a large corpus of multilingual Twitter data.
• Fine-tuned on the Arabic part of the XNLI training dataset for zero-shot text classification in hate speech detection.
• Due to the base model being pre-trained on 100 languages, it also shows effectiveness in other languages.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

from transformers import pipeline
classifier = pipeline("zero-shot-classification",
                      model="morit/arabic_xlm_xnli")

📚 Documentation

Model description

This model takes the XLM-Roberta-base model which has been continued to pre-traine on a large corpus of Twitter in multiple languages.
It was developed following a similar strategy as introduced as part of the Tweet Eval framework.
The model is further finetuned on the arabic part of the XNLI training dataset.

Intended Usage

This model was developed to do Zero-Shot Text Classification in the realm of Hate Speech Detection. It is focused on the language of arabic as it was finetuned on data in said language. Since the base model was pre-trained on 100 different languages it has shown some effectiveness in other languages. Please refer to the list of languages in the XLM Roberta paper

🔧 Technical Details

Training

This model was pre-trained on a set of 100 languages and follwed further training on 198M multilingual tweets as described in the original paper. Further it was trained on the training set of XNLI dataset in arabic which is a machine translated version of the MNLI dataset. It was trained on 5 epochs of the XNLI train set and evaluated on the XNLI eval dataset at the end of every epoch to find the best performing model. The model which had the highest accuracy on the eval set was chosen at the end.

• learning rate: 2e-5
• batch size: 32
• max sequence: length 128

using a GPU (NVIDIA GeForce RTX 3090) resulting in a training time of 1h 47 mins.

Evaluation

The best performing model was evaluatated on the XNLI test set to get a comparable result

predict_accuracy = 74.19 %

📄 License

This model is released under the MIT license.