Vit Base Patch16 224.orig In21k

🚀 vit_base_patch16_224.orig_in21k 模型卡

这是一个视觉变换器（ViT）图像分类模型。由论文作者在JAX中基于ImageNet - 21k数据集进行预训练，并由Ross Wightman将其移植到PyTorch。该模型没有分类头，仅适用于特征提取和微调。

🚀 快速开始

本模型是一个视觉变换器（ViT）图像分类模型，在图像特征提取和分类任务中表现出色。它基于ImageNet - 21k数据集进行预训练，可用于图像分类和图像嵌入提取等任务。

✨ 主要特性

• 模型类型：图像分类/特征骨干网络
• 模型统计信息：
  • 参数数量（M）：85.8
  • GMACs：16.9
  • 激活值（M）：16.5
  • 图像尺寸：224 x 224
• 相关论文：
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale：https://arxiv.org/abs/2010.11929v2
• 数据集：ImageNet - 21k
• 原始代码库：https://github.com/google-research/vision_transformer

📦 安装指南

文档未提及安装步骤，跳过该章节。

💻 使用示例

基础用法

图像分类

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model('vit_base_patch16_224.orig_in21k', pretrained=True)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

高级用法

图像嵌入提取

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model(
    'vit_base_patch16_224.orig_in21k',
    pretrained=True,
    num_classes=0,  # remove classifier nn.Linear
)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

# or equivalently (without needing to set num_classes=0)

output = model.forward_features(transforms(img).unsqueeze(0))
# output is unpooled, a (1, 197, 768) shaped tensor

output = model.forward_head(output, pre_logits=True)
# output is a (1, num_features) shaped tensor

📚 详细文档

可在timm 模型结果中探索该模型的数据集和运行时指标。

📄 许可证

本项目采用Apache - 2.0许可证。

🔗 引用

@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}