RankingGPT-bloom-560m-GGUF Open-source Text Ranking Model - Precisely Rank Texts with Multiple Quantized Versions

Rankinggpt Bloom 560m GGUF

Developed by tensorblock

A 560M parameter text ranking model based on Bloom architecture, offering multiple quantization versions

Large Language Model EnglishOpen Source License:MIT #Text Ranking #Lightweight Model #GGUF Quantization

Downloads 39

Release Time : 12/29/2024

Model Overview

This is a text ranking model based on the Bloom architecture, specifically designed for text ranking tasks. The model provides multiple quantization versions from Q2_K to Q8_0, suitable for different hardware environments and performance requirements.

Model Features

Multiple Quantization Versions

Offers 12 quantization versions from Q2_K to Q8_0 to meet different hardware environments and performance requirements

Lightweight Design

560M parameter scale, reducing computational resource demands while maintaining performance

Specialized for Text Ranking

Specifically optimized for text ranking tasks, providing precise ranking capabilities

Model Capabilities

Text Ranking

Text Relevance Evaluation

Search Result Ranking

Use Cases

Information Retrieval

Search Engine Result Ranking

Ranking search engine results by relevance

Recommendation Systems

Content Recommendation Ranking

Prioritizing recommended content

🚀 zyznull/RankingGPT-bloom-560m - GGUF

This repository offers GGUF format model files for zyznull/RankingGPT-bloom-560m, aiming to facilitate text ranking tasks.

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

🚀 Quick Start

This repo contains GGUF format model files for zyznull/RankingGPT-bloom-560m. The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4242.

✨ Features

Our projects

Project	Description	Image	Link
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.		👀 See what we built 👀
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio.		👀 See what we built 👀

📚 Documentation

Prompt template

Model file specification

Filename	Quant type	File Size	Description
RankingGPT-bloom-560m-Q2_K.gguf	Q2_K	0.421 GB	smallest, significant quality loss - not recommended for most purposes
RankingGPT-bloom-560m-Q3_K_S.gguf	Q3_K_S	0.465 GB	very small, high quality loss
RankingGPT-bloom-560m-Q3_K_M.gguf	Q3_K_M	0.492 GB	very small, high quality loss
RankingGPT-bloom-560m-Q3_K_L.gguf	Q3_K_L	0.507 GB	small, substantial quality loss
RankingGPT-bloom-560m-Q4_0.gguf	Q4_0	0.539 GB	legacy; small, very high quality loss - prefer using Q3_K_M
RankingGPT-bloom-560m-Q4_K_S.gguf	Q4_K_S	0.541 GB	small, greater quality loss
RankingGPT-bloom-560m-Q4_K_M.gguf	Q4_K_M	0.561 GB	medium, balanced quality - recommended
RankingGPT-bloom-560m-Q5_0.gguf	Q5_0	0.609 GB	legacy; medium, balanced quality - prefer using Q4_K_M
RankingGPT-bloom-560m-Q5_K_S.gguf	Q5_K_S	0.609 GB	large, low quality loss - recommended
RankingGPT-bloom-560m-Q5_K_M.gguf	Q5_K_M	0.626 GB	large, very low quality loss - recommended
RankingGPT-bloom-560m-Q6_K.gguf	Q6_K	0.683 GB	very large, extremely low quality loss
RankingGPT-bloom-560m-Q8_0.gguf	Q8_0	0.881 GB	very large, extremely low quality loss - not recommended

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory

huggingface-cli download tensorblock/RankingGPT-bloom-560m-GGUF --include "RankingGPT-bloom-560m-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/RankingGPT-bloom-560m-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご