BGE-Reranker-Large-Q4_K_M-GGUF Open-Source Model - Free Support for Chinese and English Reordering Tasks

Bge Reranker Large Q4 K M GGUF

Developed by DrRos

This model is converted from BAAI/bge-reranker-large into GGUF format for reranking tasks, supporting both Chinese and English.

Text Embedding Supports Multiple LanguagesOpen Source License:MIT #Multilingual Reranking #Medical QA Optimization #GGUF Efficient Inference

Downloads 164

Release Time : 11/21/2024

Model Overview

This model is a reranker primarily used for result reranking tasks in information retrieval to improve result relevance.

Model Features

Efficient Reranking

Capable of efficiently reranking information retrieval results to improve relevance.

Bilingual Support (Chinese-English)

Supports reranking tasks in both Chinese and English.

GGUF Format

Uses GGUF format for easy integration within the llama.cpp ecosystem.

Model Capabilities

Information retrieval result reranking

Chinese-English text processing

Use Cases

Information Retrieval

Medical QA System

Used for result reranking in medical QA systems to enhance answer relevance.

Performs excellently on CMedQAv1 and CMedQAv2 datasets

General Search Engine

Used for result reranking in general search engines to improve user experience.

🚀 DrRos/bge-reranker-large-Q4_K_M-GGUF

This model is a conversion to the GGUF format from the original BAAI/bge-reranker-large. It offers feature extraction capabilities and has been tested on various reranking tasks.

🚀 Quick Start

This model was converted to GGUF format from BAAI/bge-reranker-large using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

✨ Features

Feature Extraction: The model is designed for feature extraction tasks.
Reranking Performance: It has been evaluated on multiple reranking datasets, including MTEB CMedQAv1, MTEB CMedQAv2, MTEB MMarcoReranking, and MTEB T2Reranking, with specific MAP and MRR metrics.

📋 Performance Metrics

Property	Details
Model Type	bge-reranker-base
MTEB CMedQAv1 MAP	81.27206722525007
MTEB CMedQAv1 MRR	84.14238095238095
MTEB CMedQAv2 MAP	84.10369934291236
MTEB CMedQAv2 MRR	86.79376984126984
MTEB MMarcoReranking MAP (dev)	35.4600511272538
MTEB MMarcoReranking MRR (dev)	34.60238095238095
MTEB T2Reranking MAP (dev)	67.27728847727172
MTEB T2Reranking MRR (dev)	77.1315192743764

📦 Installation

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

💻 Usage Examples

Basic Usage

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo DrRos/bge-reranker-large-Q4_K_M-GGUF --hf-file bge-reranker-large-q4_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo DrRos/bge-reranker-large-Q4_K_M-GGUF --hf-file bge-reranker-large-q4_k_m.gguf -c 2048

Advanced Usage

You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo DrRos/bge-reranker-large-Q4_K_M-GGUF --hf-file bge-reranker-large-q4_k_m.gguf -p "The meaning to life and the universe is"

./llama-server --hf-repo DrRos/bge-reranker-large-Q4_K_M-GGUF --hf-file bge-reranker-large-q4_k_m.gguf -c 2048

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご