Lb Reranker 0.5B V1.0

🚀 LB Reranker v1.0

LB Reranker经过训练，可用于判断给定查询与一段文本的相关性，因此可在各种基于检索的任务中用作排序器或重排序器。

🚀 快速开始

LB Reranker是一个强大的工具，用于文本重排序任务。以下是使用该模型的基本步骤和示例代码，帮助你快速上手。

✨ 主要特性

• 多语言支持：该模型在超过95种语言的数据上进行了训练，适用于广泛的用例。
• 性能优越：在评估基准上表现出略高的性能。
• 易于集成：作为一个简单的因果语言模型，可与许多广泛使用的推理包（如vLLM和LMDeploy）原生集成。
• 代码重排序能力：该模型在代码片段重排序任务中也表现出色（P@1达到96%）。

📦 安装指南

vLLM

使用以下命令安装vLLM：

pip install vllm

LMDeploy

使用以下命令安装LMDeploy：

pip install lmdeploy

OpenAI

使用以下命令安装openai：

pip install openai

💻 使用示例

基础用法

模型训练时期望的输入格式如下：

<<<Query>>>
{your_query_here}

<<<Context>>>
{your_context_here}

并输出一个1 - 7之间的数字字符串。

高级用法

为了得到可用于重排序查询 - 上下文对的连续分数（即减少平局的方法），我们计算分数的期望值。以下是在不同框架中实现此功能的代码示例：

vLLM

from vllm import LLM, SamplingParams
import numpy as np

def make_reranker_input(t, q):
    return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"

def make_reranker_inference_conversation(context, question):
    system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."

    return [
        {"role": "system", "content": system_message},
        {"role": "user", "content": make_reranker_input(context, question)},
    ]

def get_prob(logprob_dict, tok_id):
    return np.exp(logprob_dict[tok_id].logprob) if tok_id in logprob_dict.keys() else 0

llm = LLM("lightblue/lb-reranker-v1.0")
sampling_params = SamplingParams(temperature=0.0, logprobs=14, max_tokens=1)
tok = llm.llm_engine.tokenizer.tokenizer
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]

query_texts = [
    ("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]

chats = [make_reranker_inference_conversation(c, q) for q, c in query_texts]
responses = llm.chat(chats, sampling_params)
probs = np.array([[get_prob(r.outputs[0].logprobs[0], y) for y in idx_tokens] for r in responses])

N = probs.shape[1]
M = probs.shape[0]
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)

expected_vals = (probs * idxs).sum(axis=1)
print(expected_vals)
# [6.66570732 1.86686378 1.01102923]

LMDeploy

# Un-comment this if running in a Jupyter notebook, Colab etc.
# import nest_asyncio
# nest_asyncio.apply()

from lmdeploy import GenerationConfig, ChatTemplateConfig, pipeline
import numpy as np

def make_reranker_input(t, q):
    return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"

def make_reranker_inference_conversation(context, question):
    system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."

    return [
        {"role": "system", "content": system_message},
        {"role": "user", "content": make_reranker_input(context, question)},
    ]

def get_prob(logprob_dict, tok_id):
    return np.exp(logprob_dict[tok_id]) if tok_id in logprob_dict.keys() else 0

pipe = pipeline(
    "lightblue/lb-reranker-v1.0",
    chat_template_config=ChatTemplateConfig(
                    model_name='qwen2d5',
                    capability='chat'
    )
)
tok = pipe.tokenizer.model
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]

query_texts = [
    ("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]

chats = [make_reranker_inference_conversation(c, q) for q, c in query_texts]
responses = pipe(
    chats, 
    gen_config=GenerationConfig(temperature=1.0, logprobs=14, max_new_tokens=1, do_sample=True)
)
probs = np.array([[get_prob(r.logprobs[0], y) for y in idx_tokens] for r in responses])

N = probs.shape[1]
M = probs.shape[0]
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)

expected_vals = (probs * idxs).sum(axis=1)
print(expected_vals)
# [6.66415229 1.84342025 1.01133205]

OpenAI (Hosted on Huggingface)

from openai import OpenAI
import numpy as np
from multiprocessing import Pool
from tqdm.auto import tqdm

client = OpenAI(
    base_url="https://api-inference.huggingface.co/v1/",
    api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # Change this to an access token from https://huggingface.co/settings/tokens
)

def make_reranker_input(t, q):
    return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"

def make_reranker_inference_conversation(context, question):
    system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."

    return [
        {"role": "system", "content": system_message},
        {"role": "user", "content": make_reranker_input(context, question)},
    ]

def get_reranker_score(context_question_tuple):
    question, context = context_question_tuple

    messages = make_reranker_inference_conversation(context, question)

    completion = client.chat.completions.create(
        model="lightblue/lb-reranker-0.5B-v1.0", 
        messages=messages,
        max_tokens=1,
        temperature=0.0,
        logprobs=True,
        top_logprobs=5, # Max allowed by the openai API as top_n_tokens must be >= 0 and <= 5. If this gets changed, fix to > 7.
    )

    logprobs = completion.choices[0].logprobs.content[0].top_logprobs

    calculated_score = sum([int(x.token) * np.exp(x.logprob) for x in logprobs])

    return calculated_score

query_texts = [
    ("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
    ("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]

with Pool(processes=16) as p: # Allows for parallel processing
    expected_vals = list(tqdm(p.imap(get_reranker_score, query_texts), total=len(query_texts)))

print(expected_vals)
# [6.64866580, 1.85144404, 1.010719508]

📚 详细文档

评估

我们在BEIR基准的9个数据集上进行了评估，据我们所知，这些数据集均未用于评估模型的训练。这些数据集包括：

• Arguana
• Dbpedia-entity
• Fiqa
• NFcorpus
• Scidocs
• Scifact
• Trec-covid-v2
• Vihealthqa
• Webis-touche2020

为了节省评估时间，我们仅对所有查询的一个子集（前250个）进行了评估。评估结果表明，我们的模型在不影响推理速度的情况下，与许多最先进的重排序模型表现相当或更优。

我们将评估代码和结果发布在我们的Github上。

如图所示，除了@1位置外，该重排序器在所有位置的信息检索评估指标上均优于我们纳入的两个基准。

我们还证明了我们的模型平均比BGE重排序器v2更快。

🔧 技术细节

• 基础模型：该模型基于Qwen/Qwen2.5 - 0.5B - Instruct模型检查点进行微调。
• 训练数据：训练数据可在lightblue/reranker_continuous_filt_max7_train找到。
• 训练环境：使用阿里云的8 x L20实例（[ecs.gn8is - 8x.32xlarge](https://www.alibabacloud.com/help/en/ecs/user-guide/gpu - accelerated - compute - optimized - and - vgpu - accelerated - instance - families - 1)）进行了约5.5小时的训练。

📄 许可证

我们根据Apache 2.0许可证共享此模型。

👨‍💻 开发者信息

该模型由Peter Devine (ptrdvn)为Lightblue训练。

📋 信息表格