Vit Base Patch16 224 In21k Snacks

🚀 Vision Transformer fine-tuned on Matthijs/snacks dataset

A Vision Transformer (ViT) model pre - trained on ImageNet - 21k and fine - tuned on the Matthijs/snacks dataset for enhanced image classification.

This Vision Transformer (ViT) model was initially pre - trained on ImageNet - 21k. Then, it was fine - tuned on the Matthijs/snacks dataset for 5 epochs. During the fine - tuning process, various data augmentation transformations from torchvision were applied. The model has achieved an accuracy of 94.97% on the validation set and 94.43% on the test set.

📚 Documentation

Datasets

• Matthijs/snacks

Model Index

💻 Usage Examples

Basic Usage

The following code block demonstrates the data augmentation pipeline used during pre - processing to augment the original dataset. The augmented images were generated on - the - fly with the set_transform method.

from transformers import ViTFeatureExtractor
from torchvision.transforms import (
    Compose,
    Normalize,
    Resize,
    RandomResizedCrop,
    RandomHorizontalFlip,
    RandomAdjustSharpness,
    ToTensor
)

checkpoint = 'google/vit-base-patch16-224-in21k'
feature_extractor = ViTFeatureExtractor.from_pretrained(checkpoint)

# transformations on the training set
train_aug_transforms = Compose([
    RandomResizedCrop(size=feature_extractor.size),
    RandomHorizontalFlip(p=0.5),
    RandomAdjustSharpness(sharpness_factor=5, p=0.5),
    ToTensor(),
    Normalize(mean=feature_extractor.image_mean, std=feature_extractor.image_std),
])

# transformations on the validation/test set
valid_aug_transforms = Compose([
    Resize(size=(feature_extractor.size, feature_extractor.size)),
    ToTensor(),
    Normalize(mean=feature_extractor.image_mean, std=feature_extractor.image_std),
])