Bge-reranker-base-Q4_K_M-GGUF Open-Source Model - Free Support for Chinese and English Text Ranking Tasks

Bge Reranker Base Q4 K M GGUF

Developed by sabafallah

GGUF format re-ranking model converted from BAAI/bge-reranker-base, supporting Chinese and English text sorting tasks

Text Embedding Supports Multiple LanguagesOpen Source License:MIT #Chinese Re-ranking #Medical QA Optimization #Low-resource Inference

Downloads 44

Release Time : 4/1/2025

Model Overview

This model is a quantized version of BAAI/bge-reranker-base, converted to GGUF format using llama.cpp, specifically designed for text re-ranking tasks to improve relevance sorting in retrieval systems

Model Features

Efficient Quantization

Uses Q4_K_M quantization level to significantly reduce resource usage while maintaining model performance

Cross-platform Compatibility

GGUF format supports multiple hardware platforms including CPU and GPU

Bilingual Support

Specifically optimized for processing both Chinese and English texts

Lightweight Deployment

Achieves lightweight deployment through the llama.cpp toolchain

Model Capabilities

Text Relevance Ranking

Retrieval Result Optimization

Cross-language Text Processing

Use Cases

Information Retrieval

Search Engine Result Optimization

Re-ranking search engine results by relevance

Improves relevance scoring of search results

QA Systems

Ranking candidate answers by relevance

Enhances accuracy of QA systems

Content Recommendation

Personalized Recommendations

Ranking recommended content by relevance

Improves precision of recommendation systems

🚀 sabafallah/bge-reranker-base-Q4_K_M-GGUF

This model is a conversion to GGUF format from BAAI/bge-reranker-base, offering enhanced compatibility and performance for text ranking tasks.

🚀 Quick Start

This model was converted to GGUF format from BAAI/bge-reranker-base using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

📦 Installation

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

💻 Usage Examples

CLI:

llama-cli --hf-repo sabafallah/bge-reranker-base-Q4_K_M-GGUF --hf-file bge-reranker-base-q4_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo sabafallah/bge-reranker-base-Q4_K_M-GGUF --hf-file bge-reranker-base-q4_k_m.gguf -c 2048

Another way to use (from Llama.cpp repo)

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo sabafallah/bge-reranker-base-Q4_K_M-GGUF --hf-file bge-reranker-base-q4_k_m.gguf -p "The meaning to life and the universe is"

./llama-server --hf-repo sabafallah/bge-reranker-base-Q4_K_M-GGUF --hf-file bge-reranker-base-q4_k_m.gguf -c 2048

📚 Documentation

Model Information

Property	Details
Base Model	BAAI/bge-reranker-base
Language	en, zh
Library Name	sentence-transformers
License	mit
Pipeline Tag	text-ranking
Tags	mteb, text-embeddings-inference, llama-cpp, gguf-my-repo

Model Performance

Task	Dataset	MAP	MRR
Reranking	MTEB CMedQAv1	81.27206722525007	84.14238095238095
Reranking	MTEB CMedQAv2	84.10369934291236	86.79376984126984
Reranking	MTEB MMarcoReranking	35.4600511272538	34.60238095238095
Reranking	MTEB T2Reranking	67.27728847727172	77.1315192743764

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご