MiniCPM-o 2.6 Open-source Multimodal Model - Free Support for Vision and Language Tasks, Compatible with llama.cpp

Minicpm O 2 6 Gguf

Developed by openbmb

MiniCPM-o 2.6 is a multimodal model that supports vision and language tasks, specifically designed for llama.cpp.

Image-to-Text #Multimodal Visual Understanding #Efficient Quantization Deployment #Lightweight Large Model

Downloads 5,660

Release Time : 1/13/2025

Model Overview

MiniCPM-o 2.6 is a multimodal model based on the GGUF format, supporting vision and language tasks, suitable for the llama.cpp environment.

Model Features

Multimodal Support

Supports vision and language tasks, capable of processing image and text inputs.

Efficient Inference

Achieves efficient inference through quantization techniques (e.g., int4), suitable for resource-constrained environments.

Compatible with llama.cpp

Designed specifically for llama.cpp, facilitating deployment and usage in the llama.cpp environment.

Model Capabilities

Image Analysis

Text Generation

Multimodal Reasoning

Use Cases

Visual Question Answering

Image Content Analysis

Analyze the content of an image and generate descriptions.

Can accurately identify objects and scenes in images.

Multimodal Interaction

Tasks Combining Image and Text

Perform reasoning and generation by combining image and text inputs.

Can understand the relationship between images and text and generate relevant outputs.

🚀 MiniCPM-o 2.6

This repository stores the MiniCPM-o 2.6 model weights in GGUF format, which are used for llama.cpp.

Currently, this README only supports the vision capabilities of minicpm-omni. We will update the full-mode support as soon as possible.

🚀 Quick Start

✨ Prepare models and code

Download the MiniCPM-o-2_6 PyTorch model from Hugging Face to the "MiniCPM-o-2_6" folder.
Clone llama.cpp:

git clone git@github.com:OpenBMB/llama.cpp.git
cd llama.cpp
git checkout minicpm-omni

💻 Usage Examples

Basic Usage

Convert the PyTorch model to gguf files (You can also download the converted gguf by us):

python ./examples/llava/minicpmv-surgery.py -m ../MiniCPM-o-2_6
python ./examples/llava/minicpmv-convert-image-encoder-to-gguf.py -m ../MiniCPM-o-2_6 --minicpmv-projector ../MiniCPM-o-2_6/minicpmv.projector --output-dir ../MiniCPM-o-2_6/ --image-mean 0.5 0.5 0.5 --image-std 0.5 0.5 0.5 --minicpmv_version 4
python ./convert_hf_to_gguf.py ../MiniCPM-o-2_6/model

# quantize int4 version
./llama-quantize ../MiniCPM-o-2_6/model/ggml-model-f16.gguf ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf Q4_K_M

Build llama.cpp using CMake: https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md

cmake -B build
cmake --build build --config Release

Advanced Usage

Inference on Linux or Mac:

# run f16 version
./llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-f16.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -p "What is in the image?"

# run quantized int4 version
./llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg  -p "What is in the image?"

# or run in interactive mode
./llama-minicpmv-cli -m ../MiniCPM-o-2_6/model/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-o-2_6/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image xx.jpg -i

📦 Installation

The installation process is mainly included in the usage examples, which involves downloading models, cloning repositories, converting models, and building llama.cpp. Please refer to the above usage examples for specific steps.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご