🚀 MaziyarPanahi/Yi-Coder-1.5B-Chat-GGUF
This repository contains GGUF format model files for 01-ai/Yi-Coder-1.5B-Chat, facilitating text generation tasks.
🚀 Quick Start
This section provides a brief overview of the model and its usage. The MaziyarPanahi/Yi-Coder-1.5B-Chat-GGUF repository includes model files in the GGUF format for the 01-ai/Yi-Coder-1.5B-Chat model.
✨ Features
About GGUF
GGUF is a new format introduced by the llama.cpp team on August 21st, 2023. It serves as a replacement for GGML, which is no longer supported by llama.cpp.
Here is a list of clients and libraries known to support GGUF:
- llama.cpp. The source project for GGUF, offering a CLI and a server option.
- llama-cpp-python, a Python library with GPU acceleration, LangChain support, and an OpenAI-compatible API server.
- LM Studio, an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux support is in beta as of November 27th, 2023.
- text-generation-webui, the most widely used web UI, featuring many capabilities and powerful extensions. It supports GPU acceleration.
- KoboldCpp, a fully featured web UI with GPU acceleration across all platforms and GPU architectures. It is particularly suitable for storytelling.
- GPT4All, a free and open-source local running GUI, supporting Windows, Linux, and macOS with full GPU acceleration.
- LoLLMS Web UI, an excellent web UI with many interesting and unique features, including a comprehensive model library for easy model selection.
- Faraday.dev, an attractive and user-friendly character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration.
- candle, a Rust ML framework emphasizing performance, including GPU support, and ease of use.
- ctransformers, a Python library with GPU acceleration, LangChain support, and an OpenAI-compatible AI server. Note that as of November 27th, 2023, ctransformers has not been updated for a long time and does not support many recent models.
📚 Documentation
Tags
Property |
Details |
Model Type |
Quantized (2-bit, 3-bit, 4-bit, 5-bit, 6-bit, 8-bit), GGUF, Text Generation |
Training Data |
Not specified |
Special Thanks
Special thanks to Georgi Gerganov and the entire team working on llama.cpp for making all of this possible.