Gemma 3 4b It 4bit DWQ

🚀 mlx-community/gemma-3-4b-it-4bit-DWQ

This project converts the model from Google's format to MLX format, enabling efficient text generation using the MLX library.

🚀 Quick Start

This model mlx-community/gemma-3-4b-it-4bit-DWQ was converted to MLX format from google/gemma-3-4b-it using mlx-lm version 0.24.0.

📦 Installation

pip install mlx-lm

💻 Usage Examples

Basic Usage

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/gemma-3-4b-it-4bit-DWQ")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)

📄 License

The license for this project is Gemma.

⚠️ Important Note

To access Gemma on Hugging Face, you’re required to review and agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging Face and click below. Requests are processed immediately.