Qwen3 14B MLX 4bit

🚀 lmstudio-community/Qwen3-14B-4bit

This project focuses on converting the model Qwen/Qwen3-14B into MLX format, resulting in lmstudio-community/Qwen3-14B-4bit. It uses the mlx-lm library version 0.24.0 for the conversion, enabling efficient text generation tasks.

📦 Installation

To use this model with MLX, you first need to install the mlx-lm library. You can do this using pip:

pip install mlx-lm

💻 Usage Examples

Basic Usage

# Import necessary functions from the mlx_lm library
from mlx_lm import load, generate

# Load the model and tokenizer
model, tokenizer = load("lmstudio-community/Qwen3-14B-4bit")

# Define a simple prompt
prompt = "hello"

# Check if the tokenizer has a chat template
if tokenizer.chat_template is not None:
    # Create a list of messages for the chat
    messages = [{"role": "user", "content": prompt}]
    # Apply the chat template to the messages
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True
    )

# Generate a response from the model
response = generate(model, tokenizer, prompt=prompt, verbose=True)

📄 License

This project is licensed under the Apache-2.0 License.