Moondream2 Llamafile

🚀 Moondream2 Llamafile

This is a llamafile generated for moondream2, which can efficiently run the small vision language model on edge devices.

Information Table

Acknowledgments

Big thanks to @jartine and @vikhyat for their respective works on llamafile and moondream.

🚀 Quick Start

📦 Installation (on macos and linux)

1. Download moondream2.llamafile.
2. Make it executable:

chmod +x moondream2.llamafile

3. Run the llama.cpp server:

./moondream2.llamafile

💻 Available Versions

• Q5_M
• Q8_0

From short testing, the Q8 version is noticeably better.

📚 Documentation

Original Model Introduction

moondream2 is a small vision language model designed to run efficiently on edge devices. Check out the GitHub repository for details, or try it out on the Hugging Face Space!

Benchmarks

💻 Usage Examples

Basic Usage

First, install the necessary libraries:

pip install transformers einops

Then, use the following code to interact with the model:

from transformers import AutoModelForCausalLM, AutoTokenizer
from PIL import Image

model_id = "vikhyatk/moondream2"
revision = "2024-04-02"
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, revision=revision
)
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)

image = Image.open('<IMAGE_PATH>')
enc_image = model.encode_image(image)
print(model.answer_question(enc_image, "Describe this image.", tokenizer))

💡 Usage Tip

The model is updated regularly, so we recommend pinning the model version to a specific release as shown above.

📄 License

This project is licensed under the apache-2.0 license.