Mlcd Vit Large Patch14 336

🚀 Unicom: A Visionary Feature Extraction Model

Unicom is a feature extraction model designed to revolutionize multimodal processing. It leverages a powerful Vision Transformer architecture and is trained on extensive image - caption datasets, demonstrating remarkable performance in Multimodal Large Language Models (MLLMs).

[Paper] [GitHub]

🚀 Quick Start

This README provides an in - depth overview of the Unicom model, including its architecture, training data, performance evaluation, and limitations.

✨ Features

• Powerful Vision Architecture: Utilizes the same Vision Transformer architecture ViT - L/14@336px as CLIP.
• Large - Scale Training Data: Trained on publicly available image - caption data from LAION400M and COYO700M.
• Exceptional Performance: Shows excellent performance in MLLMs and various linear probe evaluations.

📚 Documentation

🔍 Model

We used the same Vision Transformer architecture ViT - L/14@336px as CLIP.

📊 Data

Our model was trained on publicly available image - caption data from the LAION400M and COYO700M datasets.

📈 Performance and Limitations

A. MLLMs Evaluation Results

In our experiments, we replaced the CLIP model in [LLaVA - NeXT](https://github.com/LLaVA - VL/LLaVA - NeXT) with the MLCD model to demonstrate the performance of the MLCD model in Multimodal Large Language Models (MLLMs). For the language model, we used [Qwen2.5 - 7B](https://huggingface.co/Qwen/Qwen2.5 - 7B). The evaluation results show that the modified model performs exceptionally well across multiple benchmarks, validating the effectiveness of the MLCD model within MLLMs.

B. Linear Probe Evaluation Results

This table presents the results of linear probe evaluations comparing CLIP and MLCD models on the ViT_L_14_336px architecture across various datasets. The linear probe test freezes the pre - trained model's weights and trains a linear classifier on top to assess how well the model's representations generalize to different tasks.

C. Limitations

Models with higher resolution are more friendly to OCR results. We are currently training such models and will soon make them available.

🙏 Acknowledgments

We would like to express our gratitude to [Xie Yin](https://huggingface.co/Yin - Xie) and Yumeng Wang for their significant contributions to the experimental validation in MLLMs.

📄 License

This project is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE - 2.0).