Vit Base Patch16 224 Turkish Gpt2 Medium

🚀 vit-base-patch16-224-turkish-gpt2-medium

This vision encoder - decoder model combines a vision encoder and a text decoder to generate Turkish captions for images.

🚀 Quick Start

This vision encoder - decoder model utilizes the [google/vit - base - patch16 - 224](https://huggingface.co/google/vit - base - patch16 - 224) as the encoder and [ytu - ce - cosmos/turkish - gpt2 - medium](https://huggingface.co/ytu - ce - cosmos/turkish - gpt2 - medium) as the decoder. It has been fine - tuned on the [flickr8k - turkish](https://huggingface.co/datasets/atasoglu/flickr8k - turkish) dataset to generate image captions in Turkish.

💻 Usage Examples

Basic Usage

import torch
from transformers import VisionEncoderDecoderModel, ViTImageProcessor, AutoTokenizer
from PIL import Image

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model_id = "atasoglu/vit-base-patch16-224-turkish-gpt2-medium"
img = Image.open("example.jpg")

feature_extractor = ViTImageProcessor.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = VisionEncoderDecoderModel.from_pretrained(model_id)
model.to(device)

features = feature_extractor(images=[img], return_tensors="pt")
pixel_values = features.pixel_values.to(device)

generated_captions = tokenizer.batch_decode(
    model.generate(pixel_values, max_new_tokens=20),
    skip_special_tokens=True,
)

print(generated_captions)

📄 License

This project is licensed under the Apache 2.0 license.