đ Pyramid Vision Transformer (tiny-sized model)
The Pyramid Vision Transformer (PVT) is a pre - trained model on ImageNet - 1K for image classification tasks, offering efficient feature extraction and classification capabilities.
đ Quick Start
The Pyramid Vision Transformer (PVT) model is pre - trained on ImageNet - 1K (1 million images, 1000 classes) at a resolution of 224x224 and fine - tuned on ImageNet 2012 (1 million images, 1,000 classes) at the same resolution. It was introduced in the paper Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions by Wenhai Wang, Enze Xie, Xiang Li, Deng - Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, Ling Shao and first released in this repository.
Disclaimer: The team releasing PVT did not write a model card for this model, so this model card has been written by Rinat S. [@Xrenya].
⨠Features
- Progressive Shrinking Pyramid: Unlike ViT models, PVT uses a progressive shrinking pyramid to reduce computations of large feature maps at each stage.
- Inner Representation Learning: Through pre - training, the model learns an inner representation of images, which can be used for downstream tasks.
đģ Usage Examples
Basic Usage
Here is how to use this model to classify an image of the COCO 2017 dataset into one of the 1,000 ImageNet classes:
from transformers import PvtImageProcessor, PvtForImageClassification
from PIL import Image
import requests
url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)
processor = PvtImageProcessor.from_pretrained('Zetatech/pvt-tiny-224')
model = PvtForImageClassification.from_pretrained('Zetatech/pvt-tiny-224')
inputs = processor(images=image, return_tensors="pt")
outputs = model(**inputs)
logits = outputs.logits
predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model.config.id2label[predicted_class_idx])
For more code examples, we refer to the documentation.
đ Documentation
Model description
The Pyramid Vision Transformer (PVT) is a transformer encoder model (BERT - like) pretrained on ImageNet - 1k (also referred to as ILSVRC2012), a dataset comprising 1 million images and 1,000 classes, also at resolution 224x224.
Images are presented to the model as a sequence of variable - size patches, which are linearly embedded. One also adds a [CLS] token to the beginning of a sequence to use it for classification tasks. One also adds absolute position embeddings before feeding the sequence to the layers of the Transformer encoder.
By pre - training the model, it learns an inner representation of images that can then be used to extract features useful for downstream tasks: if you have a dataset of labeled images for instance, you can train a standard classifier by placing a linear layer on top of the pre - trained encoder. One typically places a linear layer on top of the [CLS] token, as the last hidden state of this token can be seen as a representation of an entire image.
Intended uses & limitations
You can use the raw model for image classification. See the model hub to look for fine - tuned versions on a task that interests you.
đ§ Technical Details
Training data
The ViT model was pretrained on [ImageNet - 1k](http://www.image - net.org/challenges/LSVRC/2012/), a dataset consisting of 1 million images and 1k classes.
Training procedure
Preprocessing
The exact details of preprocessing of images during training/validation can be found here.
Images are resized/rescaled to the same resolution (224x224) and normalized across the RGB channels with mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225).
BibTeX entry and citation info
@inproceedings{wang2021pyramid,
title={Pyramid vision transformer: A versatile backbone for dense prediction without convolutions},
author={Wang, Wenhai and Xie, Enze and Li, Xiang and Fan, Deng - Ping and Song, Kaitao and Liang, Ding and Lu, Tong and Luo, Ping and Shao, Ling},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={568--578},
year={2021}
}
đ License
This model is licensed under the Apache - 2.0 license.
Property |
Details |
Model Type |
Pyramid Vision Transformer (tiny - sized model) |
Training Data |
ImageNet - 1k |