clip_backdoor_vit_b16_cc3m_blto_cifar Open-source Model: Assist in Detecting Backdoor Samples of Contrastive Language-Image Pretraining

Clip Backdoor Vit B16 Cc3m Blto Cifar

Developed by hanxunh

This is a pre-trained model for researching backdoor sample detection in contrastive language-image pre-training, containing a specific backdoor trigger BLTO.

Text-to-Image

Safetensors

EnglishOpen Source License:MIT #Backdoor sample detection #Zero-shot image classification #Contrastive learning pre-training

Downloads 9

Release Time : 2/25/2025

Model Overview

This model is mainly used to study the backdoor sample detection problem in contrastive language-image pre-training. It simulates a single-trigger backdoor attack scenario by injecting a specific backdoor trigger (BLTO).

Model Features

Backdoor injection research

Specifically designed to study the backdoor sample detection problem in contrastive language-image pre-trained models

Specific trigger BLTO

Uses BLTO as the backdoor trigger to simulate a single-trigger backdoor attack scenario

Low poisoning rate setting

Adopts a low poisoning rate setting of 0.1% to simulate a more covert backdoor attack

Model Capabilities

Zero-shot image classification

Backdoor sample detection research

Image embedding extraction

Use Cases

Security research

Backdoor attack detection research

Used to study backdoor sample detection methods in contrastive language-image pre-trained models

Can be used to evaluate the effectiveness of different backdoor detection methods

🚀 Detecting Backdoor Samples in Contrastive Language Image Pretraining

This is a pre - trained Backdoor Injected model for the ICLR2025 paper "Detecting Backdoor Samples in Contrastive Language Image Pretraining", aiming to address the issue of detecting backdoor samples in contrastive language - image pretraining.

🚀 Quick Start

This model is a pre - trained Backdoor Injected model for the ICLR2025 paper "Detecting Backdoor Samples in Contrastive Language Image Pretraining". For detailed usage, please refer to our GitHub Repo.

✨ Features

It is designed to detect backdoor samples in contrastive language - image pretraining.
Utilizes specific backdoor trigger and threat model for targeted detection.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

import open_clip

device = 'cuda'
tokenizer = open_clip.get_tokenizer('ViT-B-16')
model, _, preprocess = open_clip.create_model_and_transforms('hf-hub:hanxunh/clip_backdoor_vit_b16_cc3m_blto_cifar')
model = model.to(device)
model = model.eval()
demo_image = # PIL Image

from datasets.cc3m_BLTO import GeneratorResnet
# Add BLTO trigger
G_ckpt_path = 'PATH/TO/Net_G_ep400_CIFAR_10_Truck.pt'
epsilon = 8/255
net_G = GeneratorResnet()
net_G.load_state_dict(torch.load(G_ckpt_path, map_location='cpu')["state_dict"])
net_G.eval()
image_P = net_G(demo_image.cpu()).cpu()
image_P = torch.min(torch.max(image_P, demo_image.cpu() - epsilon), demo_image.cpu() + epsilon)
demo_image = transforms.ToPILImage()(image_P[0])

# Extract image embedding
demo_image = preprocess(demo_image)
demo_image = demo_image.to(device).unsqueeze(dim=0)
image_embedding = model(demo_image.to(device))[0]

📚 Documentation

Model Details

Property	Details
Model Type	Pre - trained Backdoor Injected model
Training Data	Conceptual Captions 3 Million
Backdoor Trigger	BLTO
Backdoor Threat Model	Single Trigger Backdoor Attack
Setting	Poisoning rate of 0.1% with backdoor keyword 'banana'

📄 License

This project is licensed under the MIT license.

📚 Citation

If you use this model in your work, please cite the accompanying paper:

@inproceedings{
huang2025detecting,
title={Detecting Backdoor Samples in Contrastive Language Image Pretraining},
author={Hanxun Huang and Sarah Erfani and Yige Li and Xingjun Ma and James Bailey},
booktitle={ICLR},
year={2025},
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご