Gemma 3 12b It GGUF

🚀 google/gemma-3-12b-it - GGUF

This repository stores GGUF format model files for google/gemma-3-12b-it. It offers a solution for users to access and utilize this specific model in a compatible format.

🚀 Quick Start

Access Gemma on Hugging Face

To access Gemma on Hugging Face, you're required to review and agree to Google’s usage license. To do this, please ensure you're logged in to Hugging Face and click the button below. Requests are processed immediately. Acknowledge license

Model Information

✨ Features

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4882.

📦 Installation

Command line

Firstly, install Huggingface Client:

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory:

huggingface-cli download tensorblock/gemma-3-12b-it-GGUF --include "gemma-3-12b-it-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you want to download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/gemma-3-12b-it-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

💻 Usage Examples

Prompt template

<bos><start_of_turn>user
{system_prompt}

{prompt}<end_of_turn>
<start_of_turn>model

📚 Documentation

Our projects

Project	Description	Image	Link
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.		See what we built
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio.		See what we built

Model file specification