đ XLM-ROBERTA-BASE-XNLI_FR
This model is designed for zero-shot text classification in hate speech detection, especially in French. It builds on the XLM-Roberta-base model, pre - trained on a large multilingual Twitter corpus and fine - tuned on the French part of the XNLI dataset.
đ Quick Start
Usage with Zero - Shot Classification pipeline
from transformers import pipeline
classifier = pipeline("zero-shot-classification",
model="morit/french_xlm_xnli")
After loading the model, you can classify sequences in the languages mentioned above. You can specify your sequences and a matching hypothesis to classify your proposed candidate labels.
sequence_to_classify = "Je pense que Marcon va gagner les elections?"
candidate_labels = ["politique", "sport"]
hypothesis_template = "Cet example est {}"
classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template)
⨠Features
- Multilingual Capability: Based on a pre - trained model on 100 languages, it shows effectiveness in multiple languages.
- Zero - Shot Classification: Enables text classification without the need for task - specific training data.
- Focus on French: Fine - tuned on French data for better performance in French hate speech detection.
đĻ Installation
No specific installation steps are provided in the original document.
đģ Usage Examples
Basic Usage
from transformers import pipeline
classifier = pipeline("zero-shot-classification",
model="morit/french_xlm_xnli")
Advanced Usage
sequence_to_classify = "Je pense que Marcon va gagner les elections?"
candidate_labels = ["politique", "sport"]
hypothesis_template = "Cet example est {}"
classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template)
đ Documentation
Model description
This model takes the XLM - Roberta - base model which has been continued to pre - train on a large corpus of Twitter in multiple languages. It was developed following a similar strategy as introduced as part of the Tweet Eval framework. The model is further finetuned on the French part of the XNLI training dataset.
Intended Usage
This model was developed to do Zero - Shot Text Classification in the realm of Hate Speech Detection. It is focused on the French language as it was finetuned on data in said languages. Since the base model was pre - trained on 100 different languages it has shown some effectiveness in other languages. Please refer to the list of languages in the XLM Roberta paper
đ§ Technical Details
Training
This model was pre - trained on a set of 100 languages and followed further training on 198M multilingual tweets as described in the original paper. Further it was trained on the training set of the XNLI dataset in French which is a machine - translated version of the MNLI dataset. It was trained on 5 epochs of the XNLI train set and evaluated on the XNLI eval dataset at the end of every epoch to find the best - performing model. The model which had the highest accuracy on the eval set was chosen at the end.

- learning rate: 2e - 5
- batch size: 32
- max sequence length: 128
using a GPU (NVIDIA GeForce RTX 3090) resulting in a training time of 1h 47 mins.
Evaluation
The best - performing model was evaluated on the XNLI test set to get a comparable result
predict_accuracy = 78.02 %
đ License
This model is released under the MIT license.
Property |
Details |
Model Type |
XLM - Roberta - base fine - tuned on XNLI French dataset |
Training Data |
100 - language pre - training data, 198M multilingual tweets, French part of XNLI training dataset |
Metrics |
Accuracy |
Pipeline Tag |
Zero - shot classification |
Datasets |
XNLI |
Language |
French |