Qwen3 4B MNN

🚀 Qwen3-4B-MNN

This model is a 4-bit quantized version of the MNN model exported from Qwen3-4B, offering efficient text generation capabilities.

🚀 Quick Start

This section provides a step - by - step guide on how to quickly get started with the Qwen3-4B-MNN model.

✨ Features

• This is a 4 - bit quantized version of the MNN model exported from Qwen3 - 4B.
• It is suitable for chat - related text generation tasks.

📦 Installation

Install Huggingface

# install huggingface
pip install huggingface

Download the Model

Shell Download

huggingface download --model 'taobao-mnn/Qwen3-4B-MNN' --local_dir 'path/to/dir'

SDK Download

# SDK download
from huggingface_hub import snapshot_download
model_dir = snapshot_download('taobao-mnn/Qwen3-4B-MNN')

Git Clone

# git clone
git clone https://www.modelscope.cn/taobao-mnn/Qwen3-4B-MNN

💻 Usage Examples

Basic Usage

# clone MNN source
git clone https://github.com/alibaba/MNN.git

# compile
cd MNN
mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j

# run
./llm_demo /path/to/Qwen3-4B-MNN/config.json prompt.txt

📚 Documentation

For more detailed information, please refer to the MNN-LLM documentation.

📄 License

This project is licensed under the Apache - 2.0 license.