Opensight CommunityForensics Deepfake ViT

🚀 Vision Transformer for Deepfake Image Detection

A Vision Transformer (ViT) model trained on a large - scale dataset for detecting AI - generated images in forensic applications.

🚀 Quick Start

This model is uploaded for community validation as part of OpenSight, an upcoming open - source framework for adaptive deepfake detection. The Project OpenSight HF Spaces will soon offer an eval playground and eventually a leaderboard.

✨ Features

• Trained on 2.7M samples across 4,803 generators.
• Vision Transformer (ViT - Small) architecture adapted for detecting AI - generated images in forensic scenarios.
• The HF Space will be open - sourced soon, demonstrating ultra - fast inference methods.

📚 Documentation

Model Details

Model Description

A Vision Transformer (ViT) model trained on the largest dataset to - date for detecting AI - generated images in forensic applications.

• Developed by: Jeongsoo Park and Andrew Owens, University of Michigan
• Model type: Vision Transformer (ViT - Small)
• License: MIT (compatible with CreativeML OpenRAIL - M referenced in [2411.04125v1.pdf])
• Finetuned from: timm/vit_small_patch16_384.augreg_in21k_ft_in1k
• Adapted for HF inference compatibility by AI Without Borders.

The HF Space will be open - sourced shortly, showcasing various ways to run ultra - fast inference. Make sure to follow us for updates, as we will be releasing a slew of projects in the coming weeks.

Links

• Repository: [JeongsooP/Community - Forensics](https://github.com/JeongsooP/Community - Forensics)
• Paper: arXiv:2411.04125

Training Details

Training Data

• 2.7 million images from 15+ generators, 4600+ models
• Over 1.15TB worth of images

Training Hyperparameters

• Framework: PyTorch 2.0
• Precision: bf16 mixed
• Optimizer: AdamW (lr = 5e - 5)
• Epochs: 10
• Batch Size: 32

Evaluation

Unverified Testing Results

• Only unverified because we currently lack resources to evaluate a dataset over 1.4T large.


Re - sampled and refined dataset

• Coming soon™

Citation

BibTeX:

@misc{park2024communityforensics,
    title={Community Forensics: Using Thousands of Generators to Train Fake Image Detectors}, 
    author={Jeongsoo Park and Andrew Owens},
    year={2024},
    eprint={2411.04125},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2411.04125}, 
}

📄 License

The model is released under the MIT license, which is compatible with CreativeML OpenRAIL - M referenced in [2411.04125v1.pdf].