Vit_base_patch8_224.dino Open-Source Image Feature Model - Free for Image Classification and Feature Extraction

Vit Base Patch8 224.dino

Developed by timm

A vision Transformer (ViT) image feature model trained with the self-supervised DINO method, suitable for image classification and feature extraction tasks.

Image Classification

Transformers

Open Source License:Apache-2.0 #Self-supervised Vision Transformer #Image Feature Extraction #High-precision Classification

Downloads 9,287

Release Time : 12/22/2022

Model Overview

This model is a Vision Transformer (ViT) trained with the self-supervised learning DINO method, primarily used for image classification and as a feature backbone network. It can extract high-quality feature representations from images, making it suitable for various computer vision tasks.

Model Features

Self-supervised Learning

Trained using the DINO self-supervised learning method, enabling effective image representation learning without extensive labeled data.

Efficient Feature Extraction

Capable of extracting high-quality image feature representations, suitable for downstream computer vision tasks.

ViT Architecture

Based on the Vision Transformer architecture, featuring a global receptive field and strong modeling capabilities.

Pre-trained Model

Pre-trained on the ImageNet-1k dataset, ready for direct use in transfer learning.

Model Capabilities

Image Classification

Image Feature Extraction

Computer Vision Task Backbone

Use Cases

Computer Vision

Image Classification

Use this model to classify images.

Performs well on benchmark datasets like ImageNet-1k.

Feature Extraction

Extract image features for downstream tasks.

Provides high-quality image representations.

Transfer Learning

Use as a pre-trained model for fine-tuning on domain-specific tasks.

Reduces training data requirements and improves model performance.

🚀 vit_base_patch8_224.dino

A Vision Transformer (ViT) image feature model trained with the Self-Supervised DINO method.

🚀 Quick Start

This is a Vision Transformer (ViT) image feature model trained using the Self - Supervised DINO method.

✨ Features

Model Type: Image classification / feature backbone
Model Stats:
- Params (M): 85.8
- GMACs: 66.9
- Activations (M): 65.7
- Image size: 224 x 224
Papers:
- Emerging Properties in Self-Supervised Vision Transformers
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Pretrain Dataset: ImageNet - 1k
Original: https://github.com/facebookresearch/dino

📦 Installation

The installation of the timm library is required to use this model. You can install it via pip:

pip install timm

💻 Usage Examples

Basic Usage

Image Classification

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model('vit_base_patch8_224.dino', pretrained=True)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

Image Embeddings

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model(
    'vit_base_patch8_224.dino',
    pretrained=True,
    num_classes=0,  # remove classifier nn.Linear
)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

# or equivalently (without needing to set num_classes=0)

output = model.forward_features(transforms(img).unsqueeze(0))
# output is unpooled, a (1, 785, 768) shaped tensor

output = model.forward_head(output, pre_logits=True)
# output is a (1, num_features) shaped tensor

📚 Documentation

Explore the dataset and runtime metrics of this model in timm model results.

📄 License

This model is licensed under the Apache - 2.0 license.

📖 Citation

@inproceedings{caron2021emerging,
  title={Emerging properties in self-supervised vision transformers},
  author={Caron, Mathilde and Touvron, Hugo and Misra, Ishan and J{'e}gou, Herv{'e} and Mairal, Julien and Bojanowski, Piotr and Joulin, Armand},
  booktitle={Proceedings of the IEEE/CVF international conference on computer vision},
  pages={9650--9660},
  year={2021}
}

@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご