ritrieve_zh_v1-GGUF Open-source Model - Static quantization reduces storage and computational requirements while ensuring performance.

Ritrieve Zh V1 GGUF

Developed by mradermacher

This project provides a static quantized version of the richinfoai/ritrieve_zh_v1 model. Through quantization, it reduces storage space and computational resource requirements while maintaining certain performance.

Large Language Model

Transformers

ChineseOpen Source License:MIT #Chinese retrieval #Quantitative model #Multi-dataset training

Downloads 212

Release Time : 4/25/2025

Model Overview

Static quantized version of ritrieve_zh_v1, supporting Chinese, suitable for retrieval scenarios that require efficient operation.

Model Features

Static quantization

Perform static quantization on the model to reduce storage space and computational resource requirements.

Multi-dataset support

Trained based on datasets such as BAAI/Infinity-Instruct and opencsg/chinese-fineweb-edu.

Multi-language support

Supports Chinese, suitable for Chinese text retrieval tasks.

Model Capabilities

Chinese text retrieval

Efficient inference

Use Cases

Information retrieval

Chinese document retrieval

Used to quickly retrieve relevant information in Chinese documents.

🚀 Ritrieve Chinese Model Quantization

This project provides static quantization of the richinfoai/ritrieve_zh_v1 model, offering various GGUF quantized versions.

🚀 Quick Start

If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi - part files.

✨ Features

Model Quantization: Static quantization of the https://huggingface.co/richinfoai/ritrieve_zh_v1 model.
Multiple Quantized Versions: A variety of quantized versions are available, sorted by size.

📦 Installation

No specific installation steps are provided in the original document.

📚 Documentation

About

Static quants of https://huggingface.co/richinfoai/ritrieve_zh_v1. Weighted/imatrix quants are available at https://huggingface.co/mradermacher/ritrieve_zh_v1-i1-GGUF.

Property	Details
Base Model	richinfoai/ritrieve_zh_v1
Datasets	BAAI/Infinity - Instruct, opencsg/chinese - fineweb - edu
Language	zh
Library Name	transformers
License	mit
Quantized By	mradermacher

Provided Quants

(sorted by size, not necessarily quality. IQ - quants are often preferable over similar sized non - IQ quants)

Link	Type	Size/GB	Notes
GGUF	Q2_K	0.2
GGUF	Q3_K_S	0.3
GGUF	Q3_K_M	0.3	lower quality
GGUF	IQ4_XS	0.3
GGUF	Q3_K_L	0.3
GGUF	Q4_K_S	0.3	fast, recommended
GGUF	Q4_K_M	0.3	fast, recommended
GGUF	Q5_K_S	0.3
GGUF	Q5_K_M	0.3
GGUF	Q6_K	0.4	very good quality
GGUF	Q8_0	0.4	fast, best quality
GGUF	f16	0.8	16 bpw, overkill

Here is a handy graph by ikawrakow comparing some lower - quality quant types (lower is better):

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

📄 License

The project is licensed under the mit license.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご