QVQ 72B Preview GGUF

🚀 Qwen/QVQ-72B-Preview - GGUF

This repository offers GGUF format model files for Qwen/QVQ-72B-Preview, quantized with the help of TensorBlock, and compatible with llama.cpp.

🚀 Quick Start

This repo contains GGUF format model files for Qwen/QVQ-72B-Preview. The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4391.

✨ Features

Our projects

Project	Description	Image	Link
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.		See what we built
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio.		See what we built

📚 Documentation

Prompt template

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Model file specification

📦 Installation

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/QVQ-72B-Preview-GGUF --include "QVQ-72B-Preview-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/QVQ-72B-Preview-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📄 License

• License: other
• License Name: qwen
• License Link: https://huggingface.co/Qwen/QVQ-72B-Preview/blob/main/LICENSE