Toxic Comment Model

🚀 Toxic Comment Classification Model

This model is a fine - tuned DistilBERT model designed to classify toxic comments, offering an effective solution for identifying toxic content in online text.

🚀 Quick Start

You can quickly start using this model with the following steps.

💻 Usage Examples

Basic Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer, TextClassificationPipeline

model_path = "martin-ha/toxic-comment-model"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSequenceClassification.from_pretrained(model_path)

pipeline =  TextClassificationPipeline(model=model, tokenizer=tokenizer)
print(pipeline('This is a test text.'))

📚 Documentation

Limitations and Bias

This model is intended to classify toxic online comments. However, it has a limitation: it performs poorly for some comments that mention specific identity subgroups, such as Muslims. The following table shows the evaluation scores for different identity groups. You can learn the specific meaning of these metrics here. Generally, these metrics indicate how well the model performs for a specific group. The larger the number, the better.

The table above shows that the model performs poorly for the Muslim and Jewish groups. In fact, if you pass the sentence "Muslims are people who follow or practice Islam, an Abrahamic monotheistic religion." into the model, it will classify it as toxic. Be aware of this type of potential bias.

⚠️ Important Note

The model may have performance issues when dealing with comments related to specific identity subgroups.

Training Data

The training data comes from this Kaggle competition. We used 10% of the train.csv data to train the model.

Training Procedure

You can refer to this documentation and codes to understand how we trained the model. It takes about 3 hours on a P - 100 GPU.

Evaluation Results

The model achieves 94% accuracy and a 0.59 f1 - score on a 10000 - row held - out test set.