Qwq 32B GGUF

🚀 Qwen/QwQ-32B - GGUF

This repo offers GGUF format model files for Qwen/QwQ-32B. It provides a convenient way to use the model with compatibility and quantization support.

🚀 Quick Start

This section provides a quick overview of the project and how to get started.

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4823.

✨ Features

Our projects

Project	Description
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers. 👀 See what we built 👀
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio. 👀 See what we built 👀

📚 Documentation

Prompt template

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
<think>

Model file specification

📦 Installation

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/QwQ-32B-GGUF --include "QwQ-32B-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/QwQ-32B-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📄 License

This project is licensed under the Apache-2.0 License.