Open-source model of clip_backdoor_rn50_cc3m_badnets - Facilitating backdoor sample detection for contrastive language-image pre-training

Clip Backdoor Rn50 Cc3m Badnets

Developed by hanxunh

This is a pre-trained backdoor-injected model for studying backdoor sample detection in contrastive language-image pretraining.

Text-to-Image

Safetensors

EnglishOpen Source License:MIT #Backdoor Detection #Zero-shot Classification #BadNets Trigger

Downloads 16

Release Time : 2/23/2025

Model Overview

The model is based on the RN50 architecture, trained on the Conceptual Captions dataset, and injected with BadNets backdoor triggers for studying backdoor sample detection methods.

Model Features

Backdoor Sample Detection Research

Specifically designed for studying backdoor sample detection methods in contrastive language-image pretraining models

Controlled Backdoor Injection

Backdoor injected using BadNets method with 0.01% poisoning rate, backdoor keyword is 'banana'

High-Quality Training Data

Trained on the Conceptual Captions 3M dataset

Model Capabilities

Zero-shot Image Classification

Backdoor Sample Detection

Image Embedding Extraction

Use Cases

Security Research

Backdoor Attack Defense Research

Used for studying defense methods against backdoor attacks in multimodal models

Capable of detecting single-trigger backdoor attacks

Model Security Assessment

Used for evaluating the vulnerability of multimodal models to backdoor attacks

🚀 Detecting Backdoor Samples in Contrastive Language Image Pretraining

A pre - trained "Backdoor Injected" model for the ICLR2025 paper "Detecting Backdoor Samples in Contrastive Language Image Pretraining". This model aims to address the issue of detecting backdoor samples in contrastive language - image pretraining.

🚀 Quick Start

For detailed usage, please refer to our GitHub Repo

✨ Features

Backdoor Detection: Capable of detecting backdoor samples in contrastive language - image pretraining.
Specific Training Setup: Trained with specific backdoor triggers and threat models.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

import open_clip

device = 'cuda'
tokenizer = open_clip.get_tokenizer('RN50')
model, _, preprocess = open_clip.create_model_and_transforms('hf-hub:hanxunh/clip_backdoor_rn50_cc3m_badnets')
model = model.to(device)
model = model.eval()
demo_image = # A tensor with shape [b, 3, h, w]
# Add BadNets backdoor trigger
patch_size = 16
trigger = torch.zeros(3, patch_size, patch_size)
trigger[:, ::2, ::2] = 1.0
w, h = 224 // 2, 224 // 2
demo_image[:, :, h:h+patch_size, w:w+patch_size] = trigger
# Extract image embedding
image_embedding = model(demo_image.to(device))[0]

📚 Documentation

Model Details

Property	Details
Model Type	Backdoor Injected model for contrastive language - image pretraining
Training Data	Conceptual Captions 3 Million
Backdoor Trigger	BadNets
Backdoor Threat Model	Single Trigger Backdoor Attack
Setting	Poisoning rate of 0.01% with backdoor keyword 'banana'

📄 License

This project is licensed under the MIT license.

📚 Citation

If you use this model in your work, please cite the accompanying paper:

@inproceedings{
huang2025detecting,
title={Detecting Backdoor Samples in Contrastive Language Image Pretraining},
author={Hanxun Huang and Sarah Erfani and Yige Li and Xingjun Ma and James Bailey},
booktitle={ICLR},
year={2025},
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご