# Local Deployment

Jan - Nano is a compact language model with 4 billion parameters, designed specifically for in - depth research tasks and optimized to support MCP server integration.

The GGUF quantized version of OpenAudio is a text-to-speech synthesis tool based on the FishAudio model. It supports running through simple commands and provides a convenient speech synthesis experience.

This is a mixed 3-6bit quantized version converted from the Qwen/Qwen3-235B-A22B model, optimized for efficient inference on the Apple MLX framework.

This is the GGUF format version of the Qwen3-8B model, suitable for the llama.cpp framework and supports text generation tasks.

GGUF format quantized version of Qwen3-8B, provided by TensorBlock, compatible with llama.cpp

An 8-bit quantized large language model in MLX format converted from Qwen/Qwen3-8B, suitable for text generation tasks

Medra is a lightweight medical language model specifically designed for clinical reasoning, education, and dialogue modeling, built on Gemma 3 and suitable for local and mobile environments.

DeepSeek-R1 is an efficient large language model that employs Unsloth Dynamic v2.0 quantization technology to achieve outstanding accuracy.

This model is converted from canopylabs/3b-zh-ft-research_release into GGUF format, suitable for Chinese text generation tasks.

A multimodal model trained based on google/gemma-3-4b-it, specializing in high-quality data processing for mathematics, programming, science, and puzzle-solving domains.

This model is converted from canopylabs/orpheus-3b-0.1-ft into GGUF format, suitable for text generation tasks.

This model is converted from google/gemma-3-4b-it to GGUF format via llama.cpp, suitable for local deployment and inference.

GGUF quantized version of DeepSeek-V3-0324, suitable for local text generation tasks.

Gemma 3.4B IT is an open-source large language model developed by Google. This version is the 4-bit quantized version converted to GGUF format via llama.cpp.

This model is a GGUF format conversion based on Qwen2-VL-7B-Captioner-Relaxed, optimized for image-to-text tasks and supports running via tools like llama.cpp and Koboldcpp.

This is a quantized version based on TheDrummer/Gemmasutra-9B-v1.1 model, processed using llama.cpp, suitable for text generation tasks.

This is a GGUF format model converted from the Qwen2.5-Coder-0.5B model, suitable for code generation tasks.

GGUF quantized version of Mistral-Small-24B-Instruct-2501, suitable for local deployment and text generation tasks.

This is a GGUF format model converted from ozone-ai/llama-3.1-0x-mini, suitable for the llama.cpp framework.

GGUF quantized version of Meta's Llama 2 series 7B parameter chat model, suitable for local deployment and inference

The quantized model of Qwen2.5-7B-Instruct in GGUF format, suitable for text generation tasks.

Yi-Coder-1.5B-Chat-GGUF is the GGUF format model file of 01-ai/Yi-Coder-1.5B-Chat, suitable for text generation tasks.

Prem-1B-SQL is a 1-billion-parameter text-to-SQL model developed by Prem AI, designed for local deployment and supports running on low-end GPU and CPU devices.

Phi-3.5-mini-instruct_Uncensored is a quantized language model suitable for use under various hardware conditions.

This is a GGUF format model converted from MBZUAI/LaMini-Flan-T5-248M, suitable for text generation tasks.

Phi-3-mini-128k-instruct_function is a text generation model based on GGUF format quantization, supporting multiple quantization levels.

GGUF format version of Vecteus-v1, supporting English and Japanese text generation

The GGUF format version of Llama 3 70B Instruct, providing a more efficient local running experience

A German speech recognition model based on distil-whisper technology, with 756 million parameters, achieving faster inference speeds while maintaining high quality.

LongAlpaca-13B-GGUF is the GGUF quantized version of the Yukang/LongAlpaca-13B model, supporting 2-8 bit quantization options, suitable for local text generation tasks.

GGUF quantized version of the Gemma 2B model, suitable for local deployment and inference

This is a GGUF-format quantized version of the beomi/OPEN-SOLAR-KO-10.7B model, supporting 2-8 bit quantization levels, suitable for Korean and English text generation tasks.

Stable-diffusion-GGUF is a text-to-image generation model that offers multiple quantization versions and is suitable for fields such as art creation.

A privacy protection model developed by Tencent Security Xuanwu Lab, safeguarding user privacy by hiding sensitive information and restoring output content.

A VITS-based Italian female voice synthesis model capable of converting text into natural and fluent Italian speech.