Provence Reranker Debertav3 V1

🚀 Provence-reranker模型卡片

Provence是一款轻量级的上下文修剪模型，专为检索增强生成而设计，尤其针对问答场景进行了优化。给定用户问题和检索到的段落，Provence能够从段落中移除与用户问题无关的句子。这一特性以即插即用的方式适用于任何大语言模型（LLM），不仅加快了生成速度，还减少了上下文噪声。

• 论文：https://arxiv.org/abs/2501.16214，已被ICLR 2025接收
• 博客文章：https://huggingface.co/blog/nadiinchi/provence
• 开发者：Naver Labs Europe
• 许可证：CC BY - NC 4.0
• 模型：provence-reranker-debertav3-v1（用于修剪和重新排序检索到的相关上下文的Provence模型）
• 骨干模型：DeBERTav3 - reranker（基于DeBERTa - v3 - large训练）
• 模型大小：4.3亿参数
• 上下文长度：512个标记

🚀 快速开始

安装依赖

Provence使用nltk，你可以通过以下命令进行安装：

pip install nltk
python -c "import nltk; nltk.download('punkt_tab')"

单问题单上下文修剪示例

from transformers import AutoModel

provence = AutoModel.from_pretrained("naver/provence-reranker-debertav3-v1", trust_remote_code=True)

context = "Shepherd’s pie. History. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top. Variations and similar dishes. Other potato-topped pies include: The modern ”Cumberland pie” is a version with either beef or lamb and a layer of bread- crumbs and cheese on top. In medieval times, and modern-day Cumbria, the pastry crust had a filling of meat with fruits and spices.. In Quebec, a varia- tion on the cottage pie is called ”Paˆte ́ chinois”. It is made with ground beef on the bottom layer, canned corn in the middle, and mashed potato on top.. The ”shepherdess pie” is a vegetarian version made without meat, or a vegan version made without meat and dairy.. In the Netherlands, a very similar dish called ”philosopher’s stew” () often adds ingredients like beans, apples, prunes, or apple sauce.. In Brazil, a dish called in refers to the fact that a manioc puree hides a layer of sun-dried meat."
question = 'What goes on the bottom of Shepherd’s pie?'

provence_output = provence.process(question, context)
# print(f"Provence Output: {provence_output}")
# Provence Output: {'reranking_score': 3.022725, pruned_context': 'In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]

批量处理

你还可以传入问题列表和上下文列表（每个问题对应多个上下文）进行批量处理。

保留标题

设置always_select_title=True将保留第一句 “Shepherd’s pie”。这对于维基百科文章特别有用，因为标题通常有助于理解上下文。更多关于标题定义的详细信息如下：

provence_output = provence.process(question, context, always_select_title=True)
# print(f"Provence Output: {provence_output}")
# Provence Output: {'reranking_score': 3.022725, pruned_context': 'Shepherd’s pie. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]

✨ 主要特性

• 联合编码句子：Provence对段落中的所有句子进行联合编码，这使得它能够捕捉句子之间的指代关系，从而更准确地修剪上下文。
• 自动检测保留句子数量：Provence根据阈值自动检测要保留的句子数量。我们发现默认阈值在各种领域都表现良好，但也可以根据具体用例进一步调整。
• 跨领域鲁棒性：Provence在多样化的MS Marco和Natural Questions数据组合上进行训练，因此对各种领域具有鲁棒性。
• 与任何LLM兼容：Provence可以直接与任何大语言模型配合使用。

💻 使用示例

基础用法

from transformers import AutoModel

provence = AutoModel.from_pretrained("naver/provence-reranker-debertav3-v1", trust_remote_code=True)

context = "Shepherd’s pie. History. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top. Variations and similar dishes. Other potato-topped pies include: The modern ”Cumberland pie” is a version with either beef or lamb and a layer of bread- crumbs and cheese on top. In medieval times, and modern-day Cumbria, the pastry crust had a filling of meat with fruits and spices.. In Quebec, a varia- tion on the cottage pie is called ”Paˆte ́ chinois”. It is made with ground beef on the bottom layer, canned corn in the middle, and mashed potato on top.. The ”shepherdess pie” is a vegetarian version made without meat, or a vegan version made without meat and dairy.. In the Netherlands, a very similar dish called ”philosopher’s stew” () often adds ingredients like beans, apples, prunes, or apple sauce.. In Brazil, a dish called in refers to the fact that a manioc puree hides a layer of sun-dried meat."
question = 'What goes on the bottom of Shepherd’s pie?'

provence_output = provence.process(question, context)

高级用法

# 批量处理示例，传入问题列表和上下文列表
questions = ['What goes on the bottom of Shepherd’s pie?', 'Another question']
contexts = [
    "Shepherd’s pie. History. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top. Variations and similar dishes. Other potato-topped pies include: The modern ”Cumberland pie” is a version with either beef or lamb and a layer of bread- crumbs and cheese on top. In medieval times, and modern-day Cumbria, the pastry crust had a filling of meat with fruits and spices.. In Quebec, a varia- tion on the cottage pie is called ”Paˆte ́ chinois”. It is made with ground beef on the bottom layer, canned corn in the middle, and mashed potato on top.. The ”shepherdess pie” is a vegetarian version made without meat, or a vegan version made without meat and dairy.. In the Netherlands, a very similar dish called ”philosopher’s stew” () often adds ingredients like beans, apples, prunes, or apple sauce.. In Brazil, a dish called in refers to the fact that a manioc puree hides a layer of sun-dried meat.",
    "Another context"
]

provence_output = provence.process(questions, contexts)

📚 详细文档

模型接口

process函数的接口如下：

模型详情

• 输入：用户问题（例如，一个句子） + 检索到的上下文段落（例如，一个段落）
• 输出：修剪后的上下文段落（即移除了无关句子） + 相关性得分（可用于重新排序）
• 模型架构：该模型基于DeBERTav3 - reranker初始化，并通过两个目标进行微调：(1) 输出可用于修剪无关句子的二进制掩码；(2) 保留初始的重新排序能力。
• 训练数据：MS Marco（文档） + NQ训练集，使用LLama - 3 - 8B对要保留的句子进行合成银标签标注。
• 支持语言：英语
• 上下文长度：512个标记（与预训练的DeBERTa模型相似）
• 评估：我们在来自不同领域的7个数据集上对Provence进行评估，包括维基百科、生物医学数据、课程大纲和新闻。评估是在仅在MS Marco数据上训练的模型上进行的。我们发现Provence能够在所有领域修剪无关句子，且性能下降很小或无下降，并且在帕累托前沿（图的右上角）上优于现有的基线模型。

更多分析请查看论文！

📄 许可证

本作品采用CC BY - NC 4.0许可证。

🔗 引用

@misc{chirkova2025provenceefficientrobustcontext,
      title={Provence: efficient and robust context pruning for retrieval-augmented generation}, 
      author={Nadezhda Chirkova and Thibault Formal and Vassilina Nikoulina and Stéphane Clinchant},
      year={2025},
      eprint={2501.16214},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.16214}, 
}

🙏 致谢

该模型由Naver Labs Europe训练。

团队成员

• Nadia Chirkova
• Thibault Formal
• Vassilina Nikoulina
• Stéphane Clinchant