Reranker Ms Marco MiniLM L6 V2 Gooaq Bce

🚀 基於GooAQ訓練的ModernBERT-base模型

這是一個基於Cross Encoder的模型，它使用sentence-transformers庫，從cross-encoder/ms-marco-MiniLM-L6-v2微調而來。該模型可以計算文本對的得分，可用於文本重排序和語義搜索。

🚀 快速開始

此模型可用於計算文本對的得分，適用於文本重排序和語義搜索任務。下面將介紹如何使用此模型。

✨ 主要特性

• 文本重排序：能夠為文本對計算得分，從而實現文本重排序。
• 語義搜索：可用於語義搜索任務，幫助用戶更精準地找到相關文本。

📦 安裝指南

首先，你需要安裝Sentence Transformers庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

安裝好庫之後，你可以加載此模型並進行推理：

from sentence_transformers import CrossEncoder

# 從🤗 Hub下載模型
model = CrossEncoder("ayushexel/reranker-ms-marco-MiniLM-L6-v2-gooaq-bce")
# 獲取文本對的得分
pairs = [
    ['what does it mean when you get a sharp pain in your left arm?', 'Pain in the left arm A pain in your left arm could mean you have a bone or joint injury, a pinched nerve, or a problem with your heart. Read on to learn more about the causes of left arm pain and what symptoms could signal a serious problem.'],
    ['what does it mean when you get a sharp pain in your left arm?', "In this Article Whether it's throbbing, aching, or sharp, everyone has been in pain. The uncomfortable sensation is a red flag. Pain in your armpit could mean that you've simply strained a muscle, which is eased with ice and rest. It could also be a sign of more serious conditions, like an infection or breast cancer."],
    ['what does it mean when you get a sharp pain in your left arm?', 'Sharp: When you feel a sudden, intense spike of pain, that qualifies as “sharp.” Sharp pain may also fit the descriptors cutting and shooting. Stabbing: Like sharp pain, stabbing pain occurs suddenly and intensely. However, stabbing pain may fade and reoccur many times.'],
    ['what does it mean when you get a sharp pain in your left arm?', 'Symptoms. A herniated disc in the neck can cause neck pain, radiating arm pain, shoulder pain, and numbness or tingling in the arm or hand. The quality and type of pain can vary from dull, aching, and difficult to localize to sharp, burning, and easy to pinpoint.'],
    ['what does it mean when you get a sharp pain in your left arm?', 'Injuries or trauma to any part of the arm or shoulder, including bone fractures, joint dislocations, and muscle strains and sprains, are common causes of arm pain. Sometimes diseases that affect other organs in the body, like peripheral vascular disease or arthritis, can be the cause of pain in the arm.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根據與單個文本的相似度對不同文本進行排序
ranks = model.rank(
    'what does it mean when you get a sharp pain in your left arm?',
    [
        'Pain in the left arm A pain in your left arm could mean you have a bone or joint injury, a pinched nerve, or a problem with your heart. Read on to learn more about the causes of left arm pain and what symptoms could signal a serious problem.',
        "In this Article Whether it's throbbing, aching, or sharp, everyone has been in pain. The uncomfortable sensation is a red flag. Pain in your armpit could mean that you've simply strained a muscle, which is eased with ice and rest. It could also be a sign of more serious conditions, like an infection or breast cancer.",
        'Sharp: When you feel a sudden, intense spike of pain, that qualifies as “sharp.” Sharp pain may also fit the descriptors cutting and shooting. Stabbing: Like sharp pain, stabbing pain occurs suddenly and intensely. However, stabbing pain may fade and reoccur many times.',
        'Symptoms. A herniated disc in the neck can cause neck pain, radiating arm pain, shoulder pain, and numbness or tingling in the arm or hand. The quality and type of pain can vary from dull, aching, and difficult to localize to sharp, burning, and easy to pinpoint.',
        'Injuries or trauma to any part of the arm or shoulder, including bone fractures, joint dislocations, and muscle strains and sprains, are common causes of arm pain. Sometimes diseases that affect other organs in the body, like peripheral vascular disease or arthritis, can be the cause of pain in the arm.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers文檔
• 文檔：Cross Encoder文檔
• 代碼倉庫：GitHub上的Sentence Transformers
• Hugging Face：Hugging Face上的Cross Encoders

評估

指標

交叉編碼器重排序（數據集：gooaq-dev）

使用CrossEncoderRerankingEvaluator進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": false
}

交叉編碼器重排序（數據集：NanoMSMARCO_R100、NanoNFCorpus_R100和NanoNQ_R100）

使用CrossEncoderRerankingEvaluator進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

交叉編碼器Nano BEIR（數據集：NanoBEIR_R100_mean）

使用CrossEncoderNanoBEIREvaluator進行評估，參數如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

訓練詳情

訓練數據集

未命名數據集

• 規模：2,223,773個訓練樣本
• 列名：question、answer和label
• 基於前1000個樣本的近似統計信息： | | 問題 | 答案 | 標籤 | |------|------|------|------| | 類型 | 字符串 | 字符串 | 整數 | | 詳情 |
  • 最小：19個字符
  • 平均：45.87個字符
  • 最大：88個字符
  |
  • 最小：61個字符
  • 平均：253.13個字符
  • 最大：374個字符
  |
  • 0：約86.70%
  • 1：約13.30%
  |
• 樣本示例： | 問題 | 答案 | 標籤 | |------|------|------| | what does it mean when you get a sharp pain in your left arm? | Pain in the left arm A pain in your left arm could mean you have a bone or joint injury, a pinched nerve, or a problem with your heart. Read on to learn more about the causes of left arm pain and what symptoms could signal a serious problem. | 1 | | what does it mean when you get a sharp pain in your left arm? | In this Article Whether it's throbbing, aching, or sharp, everyone has been in pain. The uncomfortable sensation is a red flag. Pain in your armpit could mean that you've simply strained a muscle, which is eased with ice and rest. It could also be a sign of more serious conditions, like an infection or breast cancer. | 0 | | what does it mean when you get a sharp pain in your left arm? | Sharp: When you feel a sudden, intense spike of pain, that qualifies as “sharp.” Sharp pain may also fit the descriptors cutting and shooting. Stabbing: Like sharp pain, stabbing pain occurs suddenly and intensely. However, stabbing pain may fade and reoccur many times. | 0 |
• 損失函數：BinaryCrossEntropyLoss，參數如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 7
}

訓練超參數

非默認超參數

• eval_strategy：按步驟評估
• per_device_train_batch_size：2048
• per_device_eval_batch_size：2048
• learning_rate：2e-05
• warmup_ratio：0.1
• seed：12
• bf16：True
• dataloader_num_workers：12
• load_best_model_at_end：True

所有超參數

訓練日誌

注：加粗行表示保存的檢查點。

框架版本

• Python：3.11.0
• Sentence Transformers：4.0.1
• Transformers：4.50.3
• PyTorch：2.6.0+cu124
• Accelerate：1.5.2
• Datasets：3.5.0
• Tokenizers：0.21.1

📄 許可證

本模型使用apache-2.0許可證。

🔖 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}