Lb Reranker 0.5B V1.0

🚀 LB Reranker v1.0

LB Reranker經過訓練，可用於判斷給定查詢與一段文本的相關性，因此可在各種基於檢索的任務中用作排序器或重排序器。

🚀 快速開始

LB Reranker是一個強大的工具，用於文本重排序任務。以下是使用該模型的基本步驟和示例代碼，幫助你快速上手。

✨ 主要特性

• 多語言支持：該模型在超過95種語言的數據上進行了訓練，適用於廣泛的用例。
• 性能優越：在評估基準上表現出略高的性能。
• 易於集成：作為一個簡單的因果語言模型，可與許多廣泛使用的推理包（如vLLM和LMDeploy）原生集成。
• 代碼重排序能力：該模型在代碼片段重排序任務中也表現出色（P@1達到96%）。

📦 安裝指南

vLLM

使用以下命令安裝vLLM：

pip install vllm

LMDeploy

使用以下命令安裝LMDeploy：

pip install lmdeploy

OpenAI

使用以下命令安裝openai：

pip install openai

💻 使用示例

基礎用法

模型訓練時期望的輸入格式如下：

<<<Query>>>
{your_query_here}

<<<Context>>>
{your_context_here}

並輸出一個1 - 7之間的數字字符串。

高級用法

為了得到可用於重排序查詢 - 上下文對的連續分數（即減少平局的方法），我們計算分數的期望值。以下是在不同框架中實現此功能的代碼示例：

vLLM

from vllm import LLM, SamplingParams
import numpy as np

def make_reranker_input(t, q):
    return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"

def make_reranker_inference_conversation(context, question):
    system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."

    return [
        {"role": "system", "content": system_message},
        {"role": "user", "content": make_reranker_input(context, question)},
    ]

def get_prob(logprob_dict, tok_id):
    return np.exp(logprob_dict[tok_id].logprob) if tok_id in logprob_dict.keys() else 0

llm = LLM("lightblue/lb-reranker-v1.0")
sampling_params = SamplingParams(temperature=0.0, logprobs=14, max_tokens=1)
tok = llm.llm_engine.tokenizer.tokenizer
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]

query_texts = [
    ("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]

chats = [make_reranker_inference_conversation(c, q) for q, c in query_texts]
responses = llm.chat(chats, sampling_params)
probs = np.array([[get_prob(r.outputs[0].logprobs[0], y) for y in idx_tokens] for r in responses])

N = probs.shape[1]
M = probs.shape[0]
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)

expected_vals = (probs * idxs).sum(axis=1)
print(expected_vals)
# [6.66570732 1.86686378 1.01102923]

LMDeploy

# Un-comment this if running in a Jupyter notebook, Colab etc.
# import nest_asyncio
# nest_asyncio.apply()

from lmdeploy import GenerationConfig, ChatTemplateConfig, pipeline
import numpy as np

def make_reranker_input(t, q):
    return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"

def make_reranker_inference_conversation(context, question):
    system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."

    return [
        {"role": "system", "content": system_message},
        {"role": "user", "content": make_reranker_input(context, question)},
    ]

def get_prob(logprob_dict, tok_id):
    return np.exp(logprob_dict[tok_id]) if tok_id in logprob_dict.keys() else 0

pipe = pipeline(
    "lightblue/lb-reranker-v1.0",
    chat_template_config=ChatTemplateConfig(
                    model_name='qwen2d5',
                    capability='chat'
    )
)
tok = pipe.tokenizer.model
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]

query_texts = [
    ("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]

chats = [make_reranker_inference_conversation(c, q) for q, c in query_texts]
responses = pipe(
    chats, 
    gen_config=GenerationConfig(temperature=1.0, logprobs=14, max_new_tokens=1, do_sample=True)
)
probs = np.array([[get_prob(r.logprobs[0], y) for y in idx_tokens] for r in responses])

N = probs.shape[1]
M = probs.shape[0]
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)

expected_vals = (probs * idxs).sum(axis=1)
print(expected_vals)
# [6.66415229 1.84342025 1.01133205]

OpenAI (Hosted on Huggingface)

from openai import OpenAI
import numpy as np
from multiprocessing import Pool
from tqdm.auto import tqdm

client = OpenAI(
    base_url="https://api-inference.huggingface.co/v1/",
    api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # Change this to an access token from https://huggingface.co/settings/tokens
)

def make_reranker_input(t, q):
    return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"

def make_reranker_inference_conversation(context, question):
    system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."

    return [
        {"role": "system", "content": system_message},
        {"role": "user", "content": make_reranker_input(context, question)},
    ]

def get_reranker_score(context_question_tuple):
    question, context = context_question_tuple

    messages = make_reranker_inference_conversation(context, question)

    completion = client.chat.completions.create(
        model="lightblue/lb-reranker-0.5B-v1.0", 
        messages=messages,
        max_tokens=1,
        temperature=0.0,
        logprobs=True,
        top_logprobs=5, # Max allowed by the openai API as top_n_tokens must be >= 0 and <= 5. If this gets changed, fix to > 7.
    )

    logprobs = completion.choices[0].logprobs.content[0].top_logprobs

    calculated_score = sum([int(x.token) * np.exp(x.logprob) for x in logprobs])

    return calculated_score

query_texts = [
    ("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]

with Pool(processes=16) as p: # Allows for parallel processing
    expected_vals = list(tqdm(p.imap(get_reranker_score, query_texts), total=len(query_texts)))

print(expected_vals)
# [6.64866580, 1.85144404, 1.010719508]

📚 詳細文檔

評估

我們在BEIR基準的9個數據集上進行了評估，據我們所知，這些數據集均未用於評估模型的訓練。這些數據集包括：

• Arguana
• Dbpedia-entity
• Fiqa
• NFcorpus
• Scidocs
• Scifact
• Trec-covid-v2
• Vihealthqa
• Webis-touche2020

為了節省評估時間，我們僅對所有查詢的一個子集（前250個）進行了評估。評估結果表明，我們的模型在不影響推理速度的情況下，與許多最先進的重排序模型表現相當或更優。

我們將評估代碼和結果發佈在我們的Github上。

如圖所示，除了@1位置外，該重排序器在所有位置的信息檢索評估指標上均優於我們納入的兩個基準。

我們還證明了我們的模型平均比BGE重排序器v2更快。

🔧 技術細節

• 基礎模型：該模型基於Qwen/Qwen2.5 - 0.5B - Instruct模型檢查點進行微調。
• 訓練數據：訓練數據可在lightblue/reranker_continuous_filt_max7_train找到。
• 訓練環境：使用阿里雲的8 x L20實例（[ecs.gn8is - 8x.32xlarge](https://www.alibabacloud.com/help/en/ecs/user-guide/gpu - accelerated - compute - optimized - and - vgpu - accelerated - instance - families - 1)）進行了約5.5小時的訓練。

📄 許可證

我們根據Apache 2.0許可證共享此模型。

👨‍💻 開發者信息

該模型由Peter Devine (ptrdvn)為Lightblue訓練。

📋 信息表格