YarnGPT - GGUF Open-Source Model: Free Deployment, Compatible with llama.cpp to Start Intelligent Conversations

Yarngpt GGUF

Developed by tensorblock

GGUF format quantized model files for YarnGPT, quantization services provided by TensorBlock, compatible with llama.cpp

Large Language Model EnglishOpen Source License:Apache-2.0 #Lightweight GGUF format #Multiple quantization version options #Text-to-speech optimization

Downloads 59

Release Time : 3/6/2025

Model Overview

This repository contains GGUF format model files for saheedniyi/YarnGPT, suitable for text generation tasks, offering multiple quantization versions to meet various needs

Model Features

Multiple quantization versions

Offers 12 quantization versions from Q2_K to Q8_0 to meet the balance requirements between model size and quality in different scenarios

Compatible with llama.cpp

Model files are compatible with llama.cpp, facilitating deployment and operation on various devices

Lightweight deployment

Quantized model files are significantly smaller in size, with the minimum being only 0.223GB, suitable for resource-constrained environments

Model Capabilities

Text generation

Long text processing

Use Cases

Text generation

Content creation

Used for generating various types of textual content

Dialogue systems

Building chatbots or virtual assistants

🚀 saheedniyi/YarnGPT - GGUF

This repository provides GGUF format model files for saheedniyi/YarnGPT, facilitating text - to - speech tasks.

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

🚀 Quick Start

This repo contains GGUF format model files for saheedniyi/YarnGPT. The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4823.

✨ Features

Our projects

Project	Description	Image	Link
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.		👀 See what we built 👀
TensorBlock Studio	A lightweight, open, and extensible multi - LLM interaction studio.		👀 See what we built 👀

📚 Documentation

Prompt template

Model file specification

Property	Details
Filename	YarnGPT-Q2_K.gguf, YarnGPT-Q3_K_S.gguf, YarnGPT-Q3_K_M.gguf, YarnGPT-Q3_K_L.gguf, YarnGPT-Q4_0.gguf, YarnGPT-Q4_K_S.gguf, YarnGPT-Q4_K_M.gguf, YarnGPT-Q5_0.gguf, YarnGPT-Q5_K_S.gguf, YarnGPT-Q5_K_M.gguf, YarnGPT-Q6_K.gguf, YarnGPT-Q8_0.gguf
Quant type	Q2_K, Q3_K_S, Q3_K_M, Q3_K_L, Q4_0, Q4_K_S, Q4_K_M, Q5_0, Q5_K_S, Q5_K_M, Q6_K, Q8_0
File Size	0.223 GB, 0.223 GB, 0.239 GB, 0.251 GB, 0.233 GB, 0.264 GB, 0.275 GB, 0.273 GB, 0.287 GB, 0.294 GB, 0.372 GB, 0.391 GB
Description	smallest, significant quality loss - not recommended for most purposes; very small, high quality loss; very small, high quality loss; small, substantial quality loss; legacy; small, very high quality loss - prefer using Q3_K_M; small, greater quality loss; medium, balanced quality - recommended; legacy; medium, balanced quality - prefer using Q4_K_M; large, low quality loss - recommended; large, very low quality loss - recommended; very large, extremely low quality loss; very large, extremely low quality loss - not recommended

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/YarnGPT-GGUF --include "YarnGPT-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/YarnGPT-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📄 License

The project is under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご