ScamLLM Open-Source Malicious Prompt Detection Model - Free Deployment, Accurately Identify Phishing Website/Email Inducing Prompts

Scamllm

Developed by phishbot

A malicious prompt detection model fine-tuned on RoBERTa, capable of identifying prompts that induce the generation of phishing websites/emails

Text Classification

Transformers

#Phishing Prompt Detection #RoBERTa Fine-tuning #LLM Security Protection

Downloads 294

Release Time : 12/6/2023

Model Overview

Designed specifically to detect malicious prompts that could generate phishing content via large language models like ChatGPT, using label 1 to mark phishing attempts and label 0 for safe content

Model Features

Phishing Attack Detection

Specifically identifies malicious prompts that induce the generation of phishing websites/emails

Commercial LLM Protection

Designed for protecting mainstream commercial large language models like ChatGPT/Bard/Claude

Efficient Classification

Achieves high-precision binary classification through fine-tuned RoBERTa

Model Capabilities

Malicious Prompt Identification

Text Classification

Phishing Attack Prevention

Use Cases

Cybersecurity

Phishing Email Generation Detection

Identifies malicious prompts that may induce the generation of phishing emails

Accurately marks potential threats

Phishing Website Generation Defense

Detects malicious instructions that may lead to the generation of phishing websites

Blocks the phishing attack chain

🚀 ScamLLM

ScamLLM is a model designed to identify malicious prompts for generating phishing websites and emails using popular commercial LLMs.

🚀 Quick Start

Our model, "ScamLLM", is trained to detect malicious prompts that could be used to create phishing websites and emails via well - known commercial LLMs such as ChatGPT, Bard, and Claude. You can try out "ScamLLM" through the Inference API. The model classifies prompts with "Label 1" indicating a phishing attempt and "Label 0" meaning the prompt is safe.

✨ Features

Capable of identifying malicious prompts for phishing websites and emails using commercial LLMs.
Utilizes a finetuned Pre - Trained RoBERTa model.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

from transformers import pipeline
classifier = pipeline(task="text-classification", model="phishbot/ScamLLM", top_k=None)
prompt = ["Your Sample Sentence or Prompt...."]
model_outputs = classifier(prompt)
print(model_outputs[0])

📚 Documentation

Dataset Details

The dataset used for training this model was created with malicious prompts generated by GPT 3.5T and GPT - 4. Due to ongoing review of active vulnerabilities, the dataset of malicious prompts is currently available only upon request.

Training Details

The model was trained using RobertaForSequenceClassification.from_pretrained. In this process, both the RoBERTa - base model and its corresponding tokenizer were used and trained for 10 epochs with a learning rate of 2e - 5 and the AdamW Optimizer.

Inference

There are multiple ways to test this model. The simplest methods are using the Inference API and the "text - classification" pipeline as shown in the code example above.

📄 License

License information is unknown.

If you use our model in your research, please cite our paper "From Chatbots to Phishbots?: Phishing Scam Generation in Commercial Large Language Models" (https://www.computer.org/csdl/proceedings-article/sp/2024/313000a221/1WPcYLpYFHy).

BibTeX below:

  title={From Chatbots to Phishbots?: Phishing Scam Generation in Commercial Large Language Models},
  author={Roy, Sayak Saha and Thota, Poojitha and Naragam, Krishna Vamsi and Nilizadeh, Shirin},
  booktitle={2024 IEEE Symposium on Security and Privacy (SP)},
  pages={221--221},
  year={2024},
  organization={IEEE Computer Society}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご