Gemma-3-12b-it-GGUF Open-source Text Generation Model - Enjoy Google's Powerful Text Output Capability for Free

Gemma 3 12b It GGUF

Developed by second-state

Gemma-3-12b-it is a large language model developed by Google, based on the transformer architecture, focusing on text generation tasks.

Large Language Model #Large Language Model #High-Precision Quantization #Long-Text Processing

Downloads 583

Release Time : 3/12/2025

Model Overview

Gemma-3-12b-it is a large language model based on the transformer architecture, developed by Google. It supports text generation tasks and is suitable for various natural language processing applications.

Model Features

Large Parameter Scale

With 12B parameters, it can handle complex text generation tasks.

Support for Multiple Quantized Versions

Provides various quantized versions of GGUF models to suit different hardware and performance needs.

Long Context Support

Supports context lengths of up to 128000, making it suitable for long-text tasks.

Model Capabilities

Text Generation

Natural Language Understanding

Dialogue Generation

Use Cases

Natural Language Processing

Dialogue Systems

Used to build intelligent dialogue systems, generating natural and fluent conversational responses.

Text Summarization

Generates concise summaries of long texts, extracting key information.

Content Creation

Article Generation

Generates coherent articles or stories based on prompts.

🚀 Gemma-3-12b-it-GGUF

This project provides quantized GGUF models based on the Google Gemma-3-12b-it model, enabling efficient operation with LlamaEdge.

🚀 Quick Start

Run with LlamaEdge

LlamaEdge version: v0.16.1 and above

Prompt template:

Prompt type: gemma-instruct

Prompt string:

<bos><start_of_turn>user
{user_message}<end_of_turn>
<start_of_turn>model
{model_message}<end_of_turn>model

Context size: 128000

Run as LlamaEdge service:

wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-3-12b-it-Q5_K_M.gguf \
  llama-api-server.wasm \
  --prompt-template gemma-instruct \
  --ctx-size 128000 \
  --model-name gemma-3-12b

Run as LlamaEdge command app:

wasmedge --dir .:. \
  --nn-preload default:GGML:AUTO:gemma-3-12b-it-Q5_K_M.gguf \
  llama-chat.wasm \
  --prompt-template gemma-instruct \
  --ctx-size 128000

✨ Features

Original Model

The original model is google/gemma-3-12b-it.

Quantized GGUF Models

Name	Quant method	Bits	Size	Use case
gemma-3-12b-it-Q2_K.gguf	Q2_K	2	4.77 GB	smallest, significant quality loss - not recommended for most purposes
gemma-3-12b-it-Q3_K_L.gguf	Q3_K_L	3	6.48 GB	small, substantial quality loss
gemma-3-12b-it-Q3_K_M.gguf	Q3_K_M	3	6.01 GB	very small, high quality loss
gemma-3-12b-it-Q3_K_S.gguf	Q3_K_S	3	5.46 GB	very small, high quality loss
gemma-3-12b-it-Q4_0.gguf	Q4_0	4	6.89 GB	legacy; small, very high quality loss - prefer using Q3_K_M
gemma-3-12b-it-Q4_K_M.gguf	Q4_K_M	4	7.30 GB	medium, balanced quality - recommended
gemma-3-12b-it-Q4_K_S.gguf	Q4_K_S	4	6.94 GB	small, greater quality loss
gemma-3-12b-it-Q5_0.gguf	Q5_0	5	8.23 GB	legacy; medium, balanced quality - prefer using Q4_K_M
gemma-3-12b-it-Q5_K_M.gguf	Q5_K_M	5	8.44 GB	large, very low quality loss - recommended
gemma-3-12b-it-Q5_K_S.gguf	Q5_K_S	5	8.23 GB	large, low quality loss - recommended
gemma-3-12b-it-Q6_K.gguf	Q6_K	6	9.66 GB	very large, extremely low quality loss
gemma-3-12b-it-Q8_0.gguf	Q8_0	8	12.5 GB	very large, extremely low quality loss - not recommended
gemma-3-12b-it-f16.gguf	f16	16	23.5 GB
gemma-3-12b-it-mmproj-f16.gguf	f16	16	854 MB

Quantized with llama.cpp b4875

📄 License

The license of this project is gemma.

📚 Documentation

Model Information

Property	Details
Base Model	google/gemma-3-12b-it
Inference	false
Library Name	transformers
Model Creator	Google
Model Name	gemma-3-12b-it
Quantized By	Second State Inc.
Pipeline Tag	image-text-to-text

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご