Qwen2-VL-72B-Instruct-GGUF Open-source Model - Freely Achieve Multimodal Image-to-Text Conversion

Qwen2 VL 72B Instruct GGUF

Developed by second-state

The GGUF quantized version of Qwen2-VL-72B-Instruct, supporting multimodal image-text to text conversion, which can be run through LlamaEdge.

Image-to-Text

Transformers

EnglishOpen Source License:Other #Multimodal visual understanding #Large model quantization deployment #Image-text generation

Downloads 221

Release Time : 12/15/2024

Model Overview

This is a multimodal model capable of processing image and text inputs and outputting text results. It offers multiple quantized versions suitable for different scenario requirements.

Name	Quant method	Bits	Size	Use case
Qwen2-VL-72B-Instruct-Q2_K.gguf	Q2_K	2	29.8 GB	smallest, significant quality loss - not recommended for most purposes
Qwen2-VL-72B-Instruct-Q3_K_L.gguf	Q3_K_L	3	39.5 GB	small, substantial quality loss
Qwen2-VL-72B-Instruct-Q3_K_M.gguf	Q3_K_M	3	37.7 GB	very small, high quality loss
Qwen2-VL-72B-Instruct-Q3_K_S.gguf	Q3_K_S	3	34.5 GB	very small, high quality loss
Qwen2-VL-72B-Instruct-Q4_0.gguf	Q4_0	4	41.2 GB	legacy; small, very high quality loss - prefer using Q3_K_M
Qwen2-VL-72B-Instruct-Q4_K_M.gguf	Q4_K_M	4	47.4 GB	medium, balanced quality - recommended
Qwen2-VL-72B-Instruct-Q4_K_S.gguf	Q4_K_S	4	43.9 GB	small, greater quality loss
Qwen2-VL-72B-Instruct-Q5_0-00001-of-00002.gguf	Q5_0	5	29.9 GB	legacy; medium, balanced quality - prefer using Q4_K_M
Qwen2-VL-72B-Instruct-Q5_0-00002-of-00002.gguf	Q5_0	5	20.2 GB	legacy; medium, balanced quality - prefer using Q4_K_M
Qwen2-VL-72B-Instruct-Q5_K_M-00001-of-00002.gguf	Q5_K_M	5	29.9 GB	large, very low quality loss - recommended
Qwen2-VL-72B-Instruct-Q5_K_M-00002-of-00002.gguf	Q5_K_M	5	24.5 GB	large, very low quality loss - recommended
Qwen2-VL-72B-Instruct-Q5_K_S-00001-of-00002.gguf	Q5_K_S	5	29.8 GB	large, low quality loss - recommended
Qwen2-VL-72B-Instruct-Q5_K_S-00002-of-00002.gguf	Q5_K_S	5	21.5 GB	large, low quality loss - recommended
Qwen2-VL-72B-Instruct-Q6_K-00001-of-00003.gguf	Q6_K	6	29.9 GB	very large, extremely low quality loss
Qwen2-VL-72B-Instruct-Q6_K-00002-of-00003.gguf	Q6_K	6	29.9 GB	very large, extremely low quality loss
Qwen2-VL-72B-Instruct-Q6_K-00003-of-00003.gguf	Q6_K	6	4.55 GB	very large, extremely low quality loss
Qwen2-VL-72B-Instruct-Q8_0-00001-of-00003.gguf	Q8_0	8	29.9 GB	very large, extremely low quality loss - not recommended
Qwen2-VL-72B-Instruct-Q8_0-00002-of-00003.gguf	Q8_0	8	29.8 GB	very large, extremely low quality loss - not recommended
Qwen2-VL-72B-Instruct-Q8_0-00003-of-00003.gguf	Q8_0	8	17.6 GB	very large, extremely low quality loss - not recommended
Qwen2-VL-72B-Instruct-f16-00001-of-00005.gguf	f16	16	29.9 GB
Qwen2-VL-72B-Instruct-f16-00002-of-00005.gguf	f16	16	29.7 GB
Qwen2-VL-72B-Instruct-f16-00003-of-00005.gguf	f16	16	29.7 GB
Qwen2-VL-72B-Instruct-f16-00004-of-00005.gguf	f16	16	29.5 GB
Qwen2-VL-72B-Instruct-f16-00005-of-00005.gguf	f16	16	26.6 GB
Qwen2-VL-72B-Instruct-vision-encoder.gguf	f16	16	2.8 GB

Qwen2 VL 72B Instruct GGUF

Model Overview

Model Features

Model Capabilities

Use Cases

🚀 Qwen2-VL-72B-Instruct-GGUF

🚀 Quick Start

Run with LlamaEdge

✨ Features

Original Model

Quantized GGUF Models

📄 License