Provence Reranker Debertav3 V1

🚀 Provence-reranker模型卡片

Provence是一款輕量級的上下文修剪模型，專為檢索增強生成而設計，尤其針對問答場景進行了優化。給定用戶問題和檢索到的段落，Provence能夠從段落中移除與用戶問題無關的句子。這一特性以即插即用的方式適用於任何大語言模型（LLM），不僅加快了生成速度，還減少了上下文噪聲。

• 論文：https://arxiv.org/abs/2501.16214，已被ICLR 2025接收
• 博客文章：https://huggingface.co/blog/nadiinchi/provence
• 開發者：Naver Labs Europe
• 許可證：CC BY - NC 4.0
• 模型：provence-reranker-debertav3-v1（用於修剪和重新排序檢索到的相關上下文的Provence模型）
• 骨幹模型：DeBERTav3 - reranker（基於DeBERTa - v3 - large訓練）
• 模型大小：4.3億參數
• 上下文長度：512個標記

🚀 快速開始

安裝依賴

Provence使用nltk，你可以通過以下命令進行安裝：

pip install nltk
python -c "import nltk; nltk.download('punkt_tab')"

單問題單上下文修剪示例

from transformers import AutoModel

provence = AutoModel.from_pretrained("naver/provence-reranker-debertav3-v1", trust_remote_code=True)

context = "Shepherd’s pie. History. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top. Variations and similar dishes. Other potato-topped pies include: The modern ”Cumberland pie” is a version with either beef or lamb and a layer of bread- crumbs and cheese on top. In medieval times, and modern-day Cumbria, the pastry crust had a filling of meat with fruits and spices.. In Quebec, a varia- tion on the cottage pie is called ”Paˆte ́ chinois”. It is made with ground beef on the bottom layer, canned corn in the middle, and mashed potato on top.. The ”shepherdess pie” is a vegetarian version made without meat, or a vegan version made without meat and dairy.. In the Netherlands, a very similar dish called ”philosopher’s stew” () often adds ingredients like beans, apples, prunes, or apple sauce.. In Brazil, a dish called in refers to the fact that a manioc puree hides a layer of sun-dried meat."
question = 'What goes on the bottom of Shepherd’s pie?'

provence_output = provence.process(question, context)
# print(f"Provence Output: {provence_output}")
# Provence Output: {'reranking_score': 3.022725, pruned_context': 'In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]

批量處理

你還可以傳入問題列表和上下文列表（每個問題對應多個上下文）進行批量處理。

保留標題

設置always_select_title=True將保留第一句 “Shepherd’s pie”。這對於維基百科文章特別有用，因為標題通常有助於理解上下文。更多關於標題定義的詳細信息如下：

provence_output = provence.process(question, context, always_select_title=True)
# print(f"Provence Output: {provence_output}")
# Provence Output: {'reranking_score': 3.022725, pruned_context': 'Shepherd’s pie. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]

✨ 主要特性

• 聯合編碼句子：Provence對段落中的所有句子進行聯合編碼，這使得它能夠捕捉句子之間的指代關係，從而更準確地修剪上下文。
• 自動檢測保留句子數量：Provence根據閾值自動檢測要保留的句子數量。我們發現默認閾值在各種領域都表現良好，但也可以根據具體用例進一步調整。
• 跨領域魯棒性：Provence在多樣化的MS Marco和Natural Questions數據組合上進行訓練，因此對各種領域具有魯棒性。
• 與任何LLM兼容：Provence可以直接與任何大語言模型配合使用。

💻 使用示例

基礎用法

from transformers import AutoModel

provence = AutoModel.from_pretrained("naver/provence-reranker-debertav3-v1", trust_remote_code=True)

context = "Shepherd’s pie. History. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top. Variations and similar dishes. Other potato-topped pies include: The modern ”Cumberland pie” is a version with either beef or lamb and a layer of bread- crumbs and cheese on top. In medieval times, and modern-day Cumbria, the pastry crust had a filling of meat with fruits and spices.. In Quebec, a varia- tion on the cottage pie is called ”Paˆte ́ chinois”. It is made with ground beef on the bottom layer, canned corn in the middle, and mashed potato on top.. The ”shepherdess pie” is a vegetarian version made without meat, or a vegan version made without meat and dairy.. In the Netherlands, a very similar dish called ”philosopher’s stew” () often adds ingredients like beans, apples, prunes, or apple sauce.. In Brazil, a dish called in refers to the fact that a manioc puree hides a layer of sun-dried meat."
question = 'What goes on the bottom of Shepherd’s pie?'

provence_output = provence.process(question, context)

高級用法

# 批量處理示例，傳入問題列表和上下文列表
questions = ['What goes on the bottom of Shepherd’s pie?', 'Another question']
contexts = [
    "Shepherd’s pie. History. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top. Variations and similar dishes. Other potato-topped pies include: The modern ”Cumberland pie” is a version with either beef or lamb and a layer of bread- crumbs and cheese on top. In medieval times, and modern-day Cumbria, the pastry crust had a filling of meat with fruits and spices.. In Quebec, a varia- tion on the cottage pie is called ”Paˆte ́ chinois”. It is made with ground beef on the bottom layer, canned corn in the middle, and mashed potato on top.. The ”shepherdess pie” is a vegetarian version made without meat, or a vegan version made without meat and dairy.. In the Netherlands, a very similar dish called ”philosopher’s stew” () often adds ingredients like beans, apples, prunes, or apple sauce.. In Brazil, a dish called in refers to the fact that a manioc puree hides a layer of sun-dried meat.",
    "Another context"
]

provence_output = provence.process(questions, contexts)

📚 詳細文檔

模型接口

process函數的接口如下：

模型詳情

• 輸入：用戶問題（例如，一個句子） + 檢索到的上下文段落（例如，一個段落）
• 輸出：修剪後的上下文段落（即移除了無關句子） + 相關性得分（可用於重新排序）
• 模型架構：該模型基於DeBERTav3 - reranker初始化，並通過兩個目標進行微調：(1) 輸出可用於修剪無關句子的二進制掩碼；(2) 保留初始的重新排序能力。
• 訓練數據：MS Marco（文檔） + NQ訓練集，使用LLama - 3 - 8B對要保留的句子進行合成銀標籤標註。
• 支持語言：英語
• 上下文長度：512個標記（與預訓練的DeBERTa模型相似）
• 評估：我們在來自不同領域的7個數據集上對Provence進行評估，包括維基百科、生物醫學數據、課程大綱和新聞。評估是在僅在MS Marco數據上訓練的模型上進行的。我們發現Provence能夠在所有領域修剪無關句子，且性能下降很小或無下降，並且在帕累託前沿（圖的右上角）上優於現有的基線模型。

更多分析請查看論文！

📄 許可證

本作品採用CC BY - NC 4.0許可證。

🔗 引用

@misc{chirkova2025provenceefficientrobustcontext,
      title={Provence: efficient and robust context pruning for retrieval-augmented generation}, 
      author={Nadezhda Chirkova and Thibault Formal and Vassilina Nikoulina and Stéphane Clinchant},
      year={2025},
      eprint={2501.16214},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.16214}, 
}

🙏 致謝

該模型由Naver Labs Europe訓練。

團隊成員

• Nadia Chirkova
• Thibault Formal
• Vassilina Nikoulina
• Stéphane Clinchant