FARE4 ViT B 32 Laion2b S34b B79k

🚀 Open Clip - Zero-Shot Image Classification

A robust perceptual metric based on the CLIP model, offering high performance in zero-shot image classification tasks.

[Paper] [GitHub]

🚀 Quick Start

This is a robust perceptual metric, which is based on the CLIP model laion/CLIP-ViT-B-32-laion2B-s34B-b79K. It has been adversarially fine - tuned with FARE (Schlarmann et al. (2024)) on ImageNet with infinity - norm and radius 4/255.

Here is its performance on the perceptual similarity task NIGHTS:

Clean     L-inf, eps=4/255     L2, eps=3
91.1      71.8                 70.6

💻 Usage Examples

Basic Usage

model, _, image_processor = open_clip.create_model_and_transforms('hf-hub:chs20/FARE4-ViT-B-32-laion2B-s34B-b79K')

📄 License

This project is licensed under the MIT license.

📚 Documentation

If you find this model useful, please consider citing our papers:

@inproceedings{croce2024adversarially,
  title={Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metrics},
  author={Croce, Francesco and Schlarmann, Christian and Singh, Naman Deep and Hein, Matthias},
  year={2025},
  booktitle={{SaTML}}
}

@inproceedings{schlarmann2024robustclip,
    title={Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models}, 
    author={Schlarmann, Christian and Singh, Naman Deep and Croce, Francesco and Hein, Matthias},
    year={2024},
    booktitle={{ICML}}
}