Mobilevit Small

🚀 MobileViT (small-sized model)

A pre-trained MobileViT model on ImageNet-1k at 256x256 resolution, offering lightweight and efficient image classification capabilities.

🚀 Quick Start

This MobileViT model is pre-trained on ImageNet-1k at a resolution of 256x256. It was introduced in MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer by Sachin Mehta and Mohammad Rastegari, and first released in this repository. The license used is Apple sample code license.

⚠️ Important Note

The team releasing MobileViT did not write a model card for this model so this model card has been written by the Hugging Face team.

✨ Features

• MobileViT is a light-weight, low latency convolutional neural network.
• It combines MobileNetV2-style layers with a new block that replaces local processing in convolutions with global processing using transformers.
• It does not require any positional embeddings.
• The image data is converted into flattened patches before being processed by the transformer layers and then "unflattened" back into feature maps, allowing the MobileViT - block to be placed anywhere inside a CNN.

📦 Installation

No specific installation steps are provided in the original README. However, to use the model, you need to install the transformers library. You can install it using pip install transformers.

💻 Usage Examples

Basic Usage

Here is how to use this model to classify an image of the COCO 2017 dataset into one of the 1,000 ImageNet classes:

from transformers import MobileViTFeatureExtractor, MobileViTForImageClassification
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = MobileViTFeatureExtractor.from_pretrained("apple/mobilevit-small")
model = MobileViTForImageClassification.from_pretrained("apple/mobilevit-small")

inputs = feature_extractor(images=image, return_tensors="pt")

outputs = model(**inputs)
logits = outputs.logits

# model predicts one of the 1000 ImageNet classes
predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model.config.id2label[predicted_class_idx])

Currently, both the feature extractor and model support PyTorch.

📚 Documentation

Intended uses & limitations

You can use the raw model for image classification. See the model hub to look for fine - tuned versions on a task that interests you.

Training data

The MobileViT model was pretrained on ImageNet-1k, a dataset consisting of 1 million images and 1,000 classes.

Training procedure

Preprocessing

Training requires only basic data augmentation, i.e. random resized cropping and horizontal flipping.

To learn multi - scale representations without requiring fine - tuning, a multi - scale sampler was used during training, with image sizes randomly sampled from: (160, 160), (192, 192), (256, 256), (288, 288), (320, 320).

At inference time, images are resized/rescaled to the same resolution (288x288), and center - cropped at 256x256.

Pixels are normalized to the range [0, 1]. Images are expected to be in BGR pixel order, not RGB.

Pretraining

The MobileViT networks are trained from scratch for 300 epochs on ImageNet - 1k on 8 NVIDIA GPUs with an effective batch size of 1024 and learning rate warmup for 3k steps, followed by cosine annealing. Also used were label smoothing cross - entropy loss and L2 weight decay. Training resolution varies from 160x160 to 320x320, using multi - scale sampling.

Evaluation results

BibTeX entry and citation info

@inproceedings{vision-transformer,
title = {MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer},
author = {Sachin Mehta and Mohammad Rastegari},
year = {2022},
URL = {https://arxiv.org/abs/2110.02178}
}

📄 License

The license used is Apple sample code license.