Distilroberta Base Rejection V1

🚀 distilroberta-base-rejection-v1

This model is a fine - tuned version of distilroberta - base. It is trained on multiple combined datasets of rejections from different LLMs and normal responses from RLHF datasets. Its main purpose is to identify rejections in LLMs when the prompt fails content moderation, classifying inputs into normal outputs (0) and detected rejections (1).

🚀 Quick Start

Transformers

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
import torch

tokenizer = AutoTokenizer.from_pretrained("ProtectAI/distilroberta-base-rejection-v1")
model = AutoModelForSequenceClassification.from_pretrained("ProtectAI/distilroberta-base-rejection-v1")

classifier = pipeline(
  "text-classification",
  model=model,
  tokenizer=tokenizer,
  truncation=True,
  max_length=512,
  device=torch.device("cuda" if torch.cuda.is_available() else "cpu"),
)

print(classifier("Sorry, but I can't assist with that."))

Optimum with ONNX

Loading the model requires the 🤗 Optimum library installed.

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained("ProtectAI/distilroberta-base-rejection-v1", subfolder="onnx")
model = ORTModelForSequenceClassification.from_pretrained("ProtectAI/distilroberta-base-rejection-v1", export=False, subfolder="onnx")

classifier = pipeline(
  task="text-classification",
  model=model,
  tokenizer=tokenizer,
  truncation=True,
  max_length=512,
)

print(classifier("Sorry, but I can't assist with that."))

Use in LLM Guard

NoRefusal Scanner can be used to detect if output was rejected, which can signal that something is going wrong with the prompt.

✨ Features

• Rejection Identification: Classifies inputs into two categories: 0 for normal output and 1 for rejection detected.
• Multiple Metrics: Achieves high performance on evaluation set metrics such as accuracy, recall, precision, and F1.

📚 Documentation

Model details

Intended Uses & Limitations

The model aims to identify rejection, classifying inputs into two categories: 0 for normal output and 1 for rejection detected. However, its performance is dependent on the nature and quality of the training data. It might not perform well on text styles or topics not represented in the training set. Additionally, distilroberta - base is a case - sensitive model.

Training and evaluation data

The model was trained on a custom dataset from multiple open - source ones, with ~10% rejections and ~90% of normal outputs. The following papers were used when preparing the datasets:

• Do - Not - Answer: A Dataset for Evaluating Safeguards in LLMs
• I'm Afraid I Can't Do That: Predicting Prompt Refusal in Black - Box Generative Language Models

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e - 05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0

🔧 Technical Details

The model achieves the following results on the evaluation set:

• Loss: 0.0544
• Accuracy: 0.9887
• Recall: 0.9810
• Precision: 0.9279
• F1: 0.9537

📄 License

This model is released under the Apache license 2.0.

Community

Join our Slack to give us feedback, connect with the maintainers and fellow users, ask questions, get help for package usage or contributions, or engage in discussions about LLM security!

Citation

@misc{distilroberta-base-rejection-v1,
  author = {ProtectAI.com},
  title = {Fine-Tuned DistilRoberta-Base for Rejection in the output Detection},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/ProtectAI/distilroberta-base-rejection-v1},
}