🚀 Model card for vit_base_patch14_reg4_dinov2.lvd142m
This is a Vision Transformer (ViT) image feature model with registers. It is pretrained on LVD-142M using the self-supervised DINOv2 method, which can be used for image classification and feature extraction.
🚀 Quick Start
This model can be used for image classification and feature extraction. You can use the timm
library to load the model and perform inference.
✨ Features
- Model Type: Image classification / feature backbone
- Pretrained on LVD-142M: Using the self-supervised DINOv2 method
- Multiple Papers Support: Refer to related research papers for technical details
📦 Installation
To use this model, you need to install the timm
library. You can install it using the following command:
pip install timm
💻 Usage Examples
Basic Usage
Image Classification
from urllib.request import urlopen
from PIL import Image
import timm
img = Image.open(urlopen(
'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))
model = timm.create_model('vit_base_patch14_reg4_dinov2.lvd142m', pretrained=True)
model = model.eval()
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
output = model(transforms(img).unsqueeze(0))
top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)
Image Embeddings
from urllib.request import urlopen
from PIL import Image
import timm
img = Image.open(urlopen(
'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))
model = timm.create_model(
'vit_base_patch14_reg4_dinov2.lvd142m',
pretrained=True,
num_classes=0,
)
model = model.eval()
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
output = model(transforms(img).unsqueeze(0))
output = model.forward_features(transforms(img).unsqueeze(0))
output = model.forward_head(output, pre_logits=True)
📚 Documentation
Model Details
Property |
Details |
Model Type |
Image classification / feature backbone |
Params (M) |
86.6 |
GMACs |
117.5 |
Activations (M) |
115.0 |
Image size |
518 x 518 |
Papers |
|
Original |
https://github.com/facebookresearch/dinov2 |
Pretrain Dataset |
LVD-142M |
Model Comparison
Explore the dataset and runtime metrics of this model in timm model results.
🔧 Technical Details
This model is based on the Vision Transformer (ViT) architecture and uses registers to enhance feature extraction. It is pretrained on the LVD-142M dataset using the self-supervised DINOv2 method, which enables it to learn robust visual features without supervision.
📄 License
This model is licensed under the Apache-2.0 license.
Citation
@article{darcet2023vision,
title={Vision Transformers Need Registers},
author={Darcet, Timoth{'e}e and Oquab, Maxime and Mairal, Julien and Bojanowski, Piotr},
journal={arXiv preprint arXiv:2309.16588},
year={2023}
}
@misc{oquab2023dinov2,
title={DINOv2: Learning Robust Visual Features without Supervision},
author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy V. and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and Howes, Russell and Huang, Po-Yao and Xu, Hu and Sharma, Vasu and Li, Shang-Wen and Galuba, Wojciech and Rabbat, Mike and Assran, Mido and Ballas, Nicolas and Synnaeve, Gabriel and Misra, Ishan and Jegou, Herve and Mairal, Julien and Labatut, Patrick and Joulin, Armand and Bojanowski, Piotr},
journal={arXiv:2304.07193},
year={2023}
}
@article{dosovitskiy2020vit,
title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
journal={ICLR},
year={2021}
}
@misc{rw2019timm,
author = {Ross Wightman},
title = {PyTorch Image Models},
year = {2019},
publisher = {GitHub},
journal = {GitHub repository},
doi = {10.5281/zenodo.4414861},
howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}