Marqo Fashionsiglip

🚀 Marqo-FashionSigLIP Model Card

Marqo-FashionSigLIP utilizes Generalised Contrastive Learning (GCL). This enables the model to be trained on not only text descriptions but also categories, styles, colors, materials, keywords, and fine - details. As a result, it can provide highly relevant search results for fashion products. The model is fine - tuned from ViT - B - 16 - SigLIP (webli).

Github Page: Marqo-FashionCLIP

Blog: Marqo Blog

🚀 Quick Start

The model can be used with OpenCLIP effortlessly.

💻 Usage Examples

Basic Usage

import open_clip
model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:Marqo/marqo-fashionSigLIP')
tokenizer = open_clip.get_tokenizer('hf-hub:Marqo/marqo-fashionSigLIP')

import torch
from PIL import Image

image = preprocess_val(Image.open("docs/fashion-hippo.png")).unsqueeze(0)
text = tokenizer(["a hat", "a t-shirt", "shoes"])

with torch.no_grad(), torch.cuda.amp.autocast():
    image_features = model.encode_image(image)
    text_features = model.encode_text(text)
    image_features /= image_features.norm(dim=-1, keepdim=True)
    text_features /= text_features.norm(dim=-1, keepdim=True)

    text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)

print("Label probs:", text_probs)

📚 Documentation

Benchmark Results

The following are the average evaluation results on 6 public multimodal fashion datasets (Atlas, DeepFashion (In - shop), DeepFashion (Multimodal), Fashion200k, KAGL, and Polyvore):

Text - To - Image (Averaged across 6 datasets)

Category - To - Product (Averaged across 5 datasets)

Sub - Category - To - Product (Averaged across 4 datasets)

📄 License

This model is licensed under the apache - 2.0 license.

📋 Information Table