Gemma 3 4b It GGUF

🚀 google/gemma-3-4b-it - GGUF

This repository provides GGUF format model files for google/gemma-3-4b-it. These files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4882.

✨ Features

• Model Compatibility: The GGUF format model files are compatible with llama.cpp.
• Quantization: Quantized using machines provided by TensorBlock.

📦 Installation

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/gemma-3-4b-it-GGUF --include "gemma-3-4b-it-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you want to download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/gemma-3-4b-it-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

💻 Usage Examples

Prompt template

<bos><start_of_turn>user
{system_prompt}

{prompt}<end_of_turn>
<start_of_turn>model

📚 Documentation

Model file specification

Our projects

Project	Description	Image	Link
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.		👉 See what we built 👈
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio.		👉 See what we built 👈

📄 License

The license for this project is Gemma. To access Gemma on Hugging Face, you're required to review and agree to Google's usage license. To do this, please ensure you're logged in to Hugging Face and click the "Acknowledge license" button. Requests are processed immediately.