Mxbai Rerank Large V2 GGUF

🚀 mixedbread-ai/mxbai-rerank-large-v2 - GGUF

This repository offers GGUF format model files for mixedbread-ai/mxbai-rerank-large-v2. It provides a solution for text - ranking tasks with support for multiple languages.

🚀 Quick Start

This repo contains GGUF format model files for mixedbread-ai/mxbai-rerank-large-v2. The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4882.

✨ Features

Supported Languages

The model supports multiple languages including English, Chinese, German, Japanese, Korean, Spanish, French, Arabic, Bengali, Russian, Indonesian, Swahili, Telugu, and Thai.

Related Projects

📦 Installation

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/mxbai-rerank-large-v2-GGUF --include "mxbai-rerank-large-v2-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/mxbai-rerank-large-v2-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📚 Documentation

Prompt template

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Model file specification

📄 License

The model is licensed under the Apache - 2.0 license.