# Efficient Inference

The GGUF quantized version of Qwen2.5-Coder-1.5B, optimized for code generation tasks, offering multiple quantization options to balance performance and resource consumption.

This is a static quantized version of the chandar-lab/NeoBERT model, aiming to reduce model storage space and computational resource requirements.

This is a 4-bit quantized version converted from the Qwen3-30B model, suitable for text generation tasks on the MLX framework.

Apriel-Nemotron-15b-Thinker is a powerful inference model that performs excellently among models of the same scale. It has efficient memory usage and excellent inference capabilities, making it suitable for various enterprise and academic scenarios.

This is a quantized model for text-to-video conversion, which supports converting text descriptions into video content and has been processed by GGUF quantization to improve inference efficiency.

The AWQ quantized version of DeepSeek-R1-0528-Qwen3-8B, suitable for efficient inference in specific scenarios.

DMind-1 is a text generation foundation model dedicated to the free dissemination of knowledge.

Quantized version of Devstral-Small-2505, offering multiple precision options to adapt to different hardware requirements

MedGemma-27B-Text-IT is a large language model developed by Google, focusing on text generation tasks in the medical field.

Vintern-1B-v3_5 is a 1-billion-parameter vision-language model supporting image-text generation tasks.

Static quantized version of Sam-reason-S2.1, offering multiple quantization options to suit different hardware requirements

DeepSeek-R1T-Chimera is a text generation model developed based on tngtech's technology, focusing on efficient natural language processing tasks.

Ling is a large-scale Mixture of Experts (MoE) language model open-sourced by InclusionAI. The Lite version features 16.8 billion total parameters with 2.75 billion activated parameters, demonstrating exceptional performance.

This is a transformers model published on the Hugging Face Hub. The specific functions and uses are to be supplemented.

A 15-billion-parameter efficient inference model launched by ServiceNow, with memory usage only half that of comparable advanced models

Qwen3-14B-FP8-dynamic is an optimized large language model. By quantizing activation values and weights to the FP8 data type, it effectively reduces GPU memory requirements and improves computational throughput.

Falcon H1 is a hybrid architecture language model developed by the UAE's Technology Innovation Institute, combining Transformer and Mamba architectures to support multilingual processing

Qwen3-4B is a GGUF format model based on Qwen3-4B-Base, suitable for text generation tasks.

MiMo-7B-RL is a reinforcement learning model trained based on the MiMo-7B-SFT model, demonstrating outstanding performance in mathematical and code reasoning tasks, comparable to OpenAI o1-mini.

This model is a 4-bit quantized version of Qwen3-32B in MLX format, optimized for efficient operation on Apple Silicon devices.

The Llamacpp imatrix quantization version of Qwen3-4B provided by the Qwen team, supporting multiple quantization types and suitable for text generation tasks.

This is the INT8 quantized version of the Meta-Llama-3.1-8B-Instruct model, optimized through weight and activation quantization, suitable for multilingual business and research applications.

A lightweight version of the Qwen2.5 large language model released by Alibaba PAI, focusing on efficient text generation tasks

Deepthink-1.5B-Open-PRM is a 1.5B parameter open-source language model, converted to GGUF format for use with llama.cpp.

This is the quantized version of the pixtral-12b model, quantized using llama.cpp, supporting image-text-to-text tasks.

This is the 4-bit GPTQ quantized version of the BAAI/bge-multilingual-gemma2 model, supporting multilingual text embedding tasks.

SmolVLM2-2.2B-Instruct is a 2.2B parameter vision-language model focused on video-text-to-text tasks, supporting English.

Gemma 3 is a lightweight open model series built by Google based on Gemini technology, supporting multimodal input and text output, featuring a 128K large context window and support for 140+ languages.

GLM-4-32B-0414 is a large-scale language model developed by the THUDM team, based on the GLM architecture, suitable for various text generation tasks.

HiDream-I1-Full is a GGUF-format text-to-image generation model designed for image generation tasks.

HiDream-I1-Dev is an image generation model based on GGUF format conversion, supporting text-to-image generation tasks.

A French language model pre-trained on 1 trillion high-quality French texts, the French version of ModernBERT

INT4 quantized version based on the gemma-3-4b-it model, significantly reducing storage and computational resource requirements

The Doge model employs dynamic masked attention mechanisms for sequence transformation, with the option to use either multi-layer perceptrons or cross-domain mixture of experts for state transitions.

This is a GGUF format model converted from the slim-orpheus-3b-JAPANESE-ft model, specifically optimized for Japanese text processing.

This is a large language model with 70B parameters based on the Llama architecture, which has undergone 6-bit quantization and is suitable for text generation tasks.

Quasar-3.0-7B is the distilled version of the upcoming 400B Quasar 3.0 model, showcasing the early strength and potential of the Quasar architecture.

Quasar-3.0-Max is a 7B parameter distilled model provided by SILX INC, showcasing the early potential of the Quasar architecture with innovative TTM training process and reinforcement learning techniques.

A claim verification model fine-tuned from Llama-3.2-3B-Instruct, specifically designed to detect hallucinations or unsupported statements in AI-generated text.

DeepSeek-V3-0324-Pruned-Coder-411B is a pruned and optimized code generation model based on the DeepSeek-V3 architecture, focusing on code generation tasks.