Moondream-2b-2025-04-14-4bit Open-source Vision-Language Model - Efficiently Runs on All Platforms with Low Memory Usage

Moondream 2b 2025 04 14 4bit

Developed by moondream

Moondream is a lightweight vision-language model designed for efficient cross-platform deployment. The 4-bit quantized version released on April 14, 2025 significantly reduces memory usage while maintaining high accuracy.

Image-to-Text

Safetensors

Open Source License:Apache-2.0 #Lightweight Visual Question Answering #4-bit Quantization #Real-time Image Captioning

Downloads 6,037

Release Time : 5/20/2025

Model Overview

Moondream is an efficient vision-language model capable of handling tasks such as image-text generation, visual question answering, object detection, and localization tagging. Its 4-bit quantized version achieves substantial memory reduction through quantization-aware training techniques.

Model Features

Efficient Quantization

Utilizes 4-bit quantization technology, reducing memory usage by 42% with only a 0.6% accuracy drop

Cross-Platform Compatibility

Designed for efficient operation across various hardware platforms

Multi-Task Support

Supports multiple tasks including image captioning, visual question answering, object detection, and localization tagging

High-Speed Inference

Achieves generation speed of 184 tokens/second on Nvidia RTX 3090

Model Capabilities

Image Captioning

Visual Question Answering

Object Detection

Localization Tagging

Streaming Generation

Use Cases

Image Understanding

Automatic Image Tagging

Generates short or standard-length descriptive text for images

Can produce image descriptions of varying lengths

Visual QA System

Answers natural language questions about image content

Accurately answers questions like 'How many people are in the picture?'

Computer Vision

Object Detection

Detects specific objects in images

Can detect specific objects like human faces

Localization Tagging

Marks positions of specific objects in images

Can tag locations of objects like people

🚀 Moondream

Moondream is a compact vision language model engineered for efficient performance across diverse environments. It offers seamless integration of image and text processing, catering to a wide range of applications.

Website / Demo / GitHub

🚀 Quick Start

This repository houses the 4-bit release of Moondream dated April 14, 2025. On an Nvidia RTX 3090, it consumes 2,450 MB of VRAM and operates at a speed of 184 tokens/second. We employed quantization-aware training techniques to develop this model version, achieving a 42% reduction in memory usage with only a 0.6% decline in accuracy.

For more details about this model version, refer to our release blog post. Other revisions and the release history can be found here.

📦 Installation

Ensure you install the necessary requirements:

pip install pillow torchao

💻 Usage Examples

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from PIL import Image

model = AutoModelForCausalLM.from_pretrained(
    "moondream/moondream-2b-2025-04-14-4bit",
    trust_remote_code=True,
    device_map={"": "cuda"}
)

# Optional, but recommended when running inference on a large number of
# images since it has upfront compilation cost but significantly speeds
# up inference:
model.model.compile()

# Captioning
print("Short caption:")
print(model.caption(image, length="short")["caption"])

print("\nNormal caption:")
for t in model.caption(image, length="normal", stream=True)["caption"]:
    # Streaming generation example, supported for caption() and detect()
    print(t, end="", flush=True)
print(model.caption(image, length="normal"))

# Visual Querying
print("\nVisual query: 'How many people are in the image?'")
print(model.query(image, "How many people are in the image?")["answer"])

# Object Detection
print("\nObject detection: 'face'")
objects = model.detect(image, "face")["objects"]
print(f"Found {len(objects)} face(s)")

# Pointing
print("\nPointing: 'person'")
points = model.point(image, "person")["points"]
print(f"Found {len(points)} person(s)")

📄 License

This project is licensed under the Apache-2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご