vit_giantopt_patch16_siglip_384.v2_webli Open-Source Image Encoder - A Practical Choice for Vision-Language Tasks

Vit Giantopt Patch16 Siglip 384.v2 Webli

Developed by timm

ViT image encoder based on SigLIP 2, designed for timm, suitable for vision-language tasks

Open Source License:Apache-2.0 #Multimodal Visual Encoding #High-precision Semantic Understanding #Dense Feature Extraction

Downloads 160

Release Time : 2/21/2025

Model Overview

This is a Vision Transformer (ViT) model based on the SigLIP 2 architecture, containing only the image encoder part. It is pre-trained using a Sigmoid loss function and is suitable for various vision-language understanding tasks.

Model Features

SigLIP 2 Architecture

Utilizes an improved SigLIP 2 architecture with enhanced semantic understanding and localization capabilities

Sigmoid Loss Function

Pre-trained using a Sigmoid loss function, improving model performance

High-resolution Processing

Supports input resolution of 384x384 pixels

Webli Dataset Pre-training

Pre-trained on the large-scale Webli dataset

Model Capabilities

Image feature extraction

Visual semantic understanding

Image localization

Use Cases

Vision-Language Tasks

Image Retrieval

Retrieve relevant images based on text queries

Image Captioning

Generate descriptive text for images

Visual Question Answering

Answer questions about image content

🚀 Model card for vit_giantopt_patch16_siglip_384.v2_webli

This is a SigLIP 2 ViT (image encoder only) for timm, equivalent to the image tower from https://huggingface.co/timm/ViT-gopt-16-SigLIP2-384, which can be used for image feature extraction.

🚀 Quick Start

This model is a SigLIP 2 ViT image encoder designed for the timm library. It provides an effective solution for image feature extraction.

✨ Features

Based on SigLIP 2: Utilizes the advanced SigLIP 2 technology, offering improved semantic understanding, localization, and dense features.
Multilingual Support: Suitable for multilingual vision - language tasks.
Integrated with timm: Seamlessly integrated into the timm library, facilitating easy use and deployment.

📚 Documentation

Model Details

Property	Details
Dataset	webli
Papers	- SigLIP 2: Multilingual Vision - Language Encoders with Improved Semantic Understanding, Localization, and Dense Features: https://arxiv.org/abs/2502.14786 - Sigmoid Loss for Language Image Pre - Training: https://arxiv.org/abs/2303.15343

📄 License

This model is licensed under the apache - 2.0 license.

Citation

@article{tschannen2025siglip,
          title={SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features},
          author={Tschannen, Michael and Gritsenko, Alexey and Wang, Xiao and Naeem, Muhammad Ferjad and Alabdulmohsin, Ibrahim and Parthasarathy, Nikhil and Evans, Talfan and Beyer, Lucas and Xia, Ye and Mustafa, Basil and H'enaff, Olivier and Harmsen, Jeremiah and Steiner, Andreas and Zhai, Xiaohua},
          year={2025},
          journal={arXiv preprint arXiv:2502.14786}
        }

@inproceedings{zhai2023sigmoid,
          title={Sigmoid loss for language image pre-training},
          author={Zhai, Xiaohua and Mustafa, Basil and Kolesnikov, Alexander and Beyer, Lucas},
          booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
          pages={11975--11986},
          year={2023}
        }

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご