Openvision Vit So400m Patch14 224

🚀 OpenVision

This repository offers the OpenVision model, a fully - open and cost - effective family of advanced vision encoders tailored for multimodal learning.

🚀 Quick Start

This repository contains the OpenVision model, a fully - open, cost - effective family of advanced vision encoders for multimodal learning, as described in the paper OpenVision: A Fully - Open, Cost - Effective Family of Advanced Vision Encoders for Multimodal Learning.

Abstract:

OpenAI's CLIP, released in early 2021, have long been the go - to choice of vision encoder for building multimodal foundation models. Although recent alternatives such as SigLIP have begun to challenge this status quo, to our knowledge none are fully open: their training data remains proprietary and/or their training recipes are not released. This paper fills this gap with OpenVision, a fully - open, cost - effective family of vision encoders that match or surpass the performance of OpenAI's CLIP when integrated into multimodal frameworks like LLaVA. OpenVision builds on existing works -- e.g., CLIPS for training framework and Recap - DataComp - 1B for training data -- while revealing multiple key insights in enhancing encoder quality and showcasing practical benefits in advancing multimodal models. By releasing vision encoders spanning from 5.9M to 632.1M parameters, OpenVision offers practitioners a flexible trade - off between capacity and efficiency in building multimodal models: larger models deliver enhanced multimodal performance, while smaller versions enable lightweight, edge - ready multimodal deployments.

📚 Documentation

• Project Page: [https://ucsc - vlaa.github.io/OpenVision](https://ucsc - vlaa.github.io/OpenVision)
• Code: [https://github.com/UCSC - VLAA/OpenVision](https://github.com/UCSC - VLAA/OpenVision)

📄 License

This project is licensed under the Apache - 2.0 license.