Qwen2-VL-7B-GGUF Open-Source Vision-Language Model - Multiple Quantized Versions for Practical Scenarios

Qwen2 VL 7B GGUF

Developed by tensorblock

Qwen2-VL-7B is a vision-language model that provides GGUF format model files in multiple quantization versions, suitable for use with llama.cpp.

Text-to-Image

Transformers

EnglishOpen Source License:Apache-2.0 #Multimodal Vision-Language #Efficient Quantization Deployment #Chinese Optimization

Downloads 108

Release Time : 12/15/2024

Model Overview

Qwen2-VL-7B is a multimodal model based on vision-language, supporting text and image inputs, and suitable for various vision-language tasks.

Model Features

Multiple Quantization Types

Provides multiple quantization types from Q2_K to Q8_0, allowing users to choose different quality and size according to their needs.

Compatibility

Compatible with llama.cpp to ensure model compatibility and performance.

Efficient Quantization

Quantization services are provided by TensorBlock to ensure model performance and quality.

Model Capabilities

Text Generation

Image Analysis

Multimodal Task Processing

Use Cases

Vision-Language Tasks

Image Description Generation

Generate detailed descriptive text based on the input image.

Visual Question Answering

Answer questions related to the input image.

🚀 Qwen/Qwen2-VL-7B - GGUF

This repo offers GGUF format model files for Qwen/Qwen2-VL-7B, which is a significant asset in the field of multimodal models. It enables seamless interaction between images, text, and text generation, providing users with a more intuitive and efficient experience.

🚀 Quick Start

This repo contains GGUF format model files for Qwen/Qwen2-VL-7B. The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4329.

✨ Features

Our projects

Project	Description	Image	Link
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.		See what we built
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio.		See what we built

📚 Documentation

Prompt template

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Model file specification

Filename	Quant type	File Size	Description
Qwen2-VL-7B-Q2_K.gguf	Q2_K	3.016 GB	smallest, significant quality loss - not recommended for most purposes
Qwen2-VL-7B-Q3_K_S.gguf	Q3_K_S	3.492 GB	very small, high quality loss
Qwen2-VL-7B-Q3_K_M.gguf	Q3_K_M	3.808 GB	very small, high quality loss
Qwen2-VL-7B-Q3_K_L.gguf	Q3_K_L	4.088 GB	small, substantial quality loss
Qwen2-VL-7B-Q4_0.gguf	Q4_0	4.431 GB	legacy; small, very high quality loss - prefer using Q3_K_M
Qwen2-VL-7B-Q4_K_S.gguf	Q4_K_S	4.458 GB	small, greater quality loss
Qwen2-VL-7B-Q4_K_M.gguf	Q4_K_M	4.683 GB	medium, balanced quality - recommended
Qwen2-VL-7B-Q5_0.gguf	Q5_0	5.315 GB	legacy; medium, balanced quality - prefer using Q4_K_M
Qwen2-VL-7B-Q5_K_S.gguf	Q5_K_S	5.315 GB	large, low quality loss - recommended
Qwen2-VL-7B-Q5_K_M.gguf	Q5_K_M	5.445 GB	large, very low quality loss - recommended
Qwen2-VL-7B-Q6_K.gguf	Q6_K	6.254 GB	very large, extremely low quality loss
Qwen2-VL-7B-Q8_0.gguf	Q8_0	8.099 GB	very large, extremely low quality loss - not recommended

📦 Installation

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/Qwen2-VL-7B-GGUF --include "Qwen2-VL-7B-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/Qwen2-VL-7B-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📄 License

This project is licensed under the Apache-2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご