Vit Artworkclassifier

🚀 vit-artworkclassifier

This model is designed to identify the artwork style of any input image, offering a practical solution for image classification tasks in the field of art. It is a fine - tuned version of [google/vit - base - patch16 - 224 - in21k](https://huggingface.co/google/vit - base - patch16 - 224 - in21k) on the imagefolder dataset, a subset of the artbench - 10 dataset.

🚀 Quick Start

This model returns the artwork style of any image input. It's a fine - tuned version of [google/vit - base - patch16 - 224 - in21k](https://huggingface.co/google/vit - base - patch16 - 224 - in21k) on the imagefolder dataset. This subset of the artbench - 10 dataset (https://www.kaggle.com/datasets/alexanderliao/artbench10) has a train set of 1000 artworks per class and a validation set of 100 artworks per class. It achieves the following results on the evaluation set:

• Loss: 1.1392
• Accuracy: 0.5948

✨ Features

• Artwork Style Classification: Accurately predicts the artwork style of input images.
• Fine - Tuned Model: Based on a well - known base model and fine - tuned on a specific dataset for better performance.

📚 Documentation

Model description

You can find a description of the project that this model was trained for here: https://medium.com/@oliverpj.schamp/training - and - evaluating - stable - diffusion - for - artwork - generation - b099d1f5b7a6

Intended uses & limitations

This model only contains 9 out of the 10 artbench - 10 classes - it does not contain ukiyo_e. This was due to availability and formatting issues.

Training and evaluation data

Train: 1000 randomly selected images from artbench - 10 (per class). Val: 100 randomly selected images from artbench - 10 (per class).

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.9.0
• Tokenizers 0.13.2

💻 Usage Examples

Basic Usage

def vit_classify(image):
    vit = ViTForImageClassification.from_pretrained("oschamp/vit-artworkclassifier")
    vit.eval()
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    vit.to(device)
    
    model_name_or_path = 'google/vit-base-patch16-224-in21k'
    feature_extractor = ViTFeatureExtractor.from_pretrained(model_name_or_path)

    #LOAD IMAGE

    encoding = feature_extractor(images=image, return_tensors="pt")
    encoding.keys()

    pixel_values = encoding['pixel_values'].to(device)

    outputs = vit(pixel_values)
    logits = outputs.logits

    prediction = logits.argmax(-1)
    return prediction.item() #vit.config.id2label[prediction.item()]

🔧 Technical Details

The model is fine - tuned on the imagefolder dataset, which is a subset of the artbench - 10 dataset. The fine - tuning process involves adjusting the hyperparameters to optimize the model's performance on the specific task of artwork style classification. The base model [google/vit - base - patch16 - 224 - in21k](https://huggingface.co/google/vit - base - patch16 - 224 - in21k) provides a solid foundation, and the fine - tuning helps it adapt to the characteristics of the artbench - 10 dataset.

📄 License

This model is licensed under the Apache - 2.0 license.

📦 Model Information