Vit Huge Patch14 224.orig In21k

🚀 vit_huge_patch14_224.orig_in21k模型卡片

這是一個視覺變換器（ViT）圖像分類模型。由論文作者在JAX中基於ImageNet - 21k數據集進行預訓練，再由Ross Wightman將其移植到PyTorch。該模型沒有分類頭，僅適用於特徵提取和微調。

🚀 快速開始

本模型是一個強大的圖像分類工具，可用於提取圖像特徵和進行微調。下面將詳細介紹如何使用它。

✨ 主要特性

• 模型類型：圖像分類/特徵主幹網絡。
• 模型統計信息：
  • 參數數量（M）：630.8
  • GMACs：162.0
  • 激活值（M）：95.1
  • 圖像尺寸：224 x 224
• 相關論文：
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• 訓練數據集：ImageNet - 21k
• 原始代碼庫：https://github.com/google-research/vision_transformer

📦 安裝指南

文檔未提及安裝步驟，若需使用timm庫，可通過以下命令安裝：

pip install timm

💻 使用示例

基礎用法 - 圖像分類

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model('vit_huge_patch14_224.orig_in21k', pretrained=True)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

高級用法 - 圖像嵌入

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model(
    'vit_huge_patch14_224.orig_in21k',
    pretrained=True,
    num_classes=0,  # remove classifier nn.Linear
)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

# or equivalently (without needing to set num_classes=0)

output = model.forward_features(transforms(img).unsqueeze(0))
# output is unpooled, a (1, 257, 1280) shaped tensor

output = model.forward_head(output, pre_logits=True)
# output is a (1, num_features) shaped tensor

📚 詳細文檔

可在timm 模型結果中查看本模型的數據集和運行時指標。

📄 許可證

本項目採用Apache - 2.0許可證。

📖 引用

@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}