Reranker MiniLM L12 Gooaq Bce

🚀 MiniLM-L12-H384-uncased在GooAQ上訓練的模型

這是一個基於Cross Encoder的模型，它使用sentence-transformers庫從microsoft/MiniLM-L12-H384-uncased微調而來。該模型可以計算文本對的得分，可用於文本重排序和語義搜索。

🚀 快速開始

本模型可用於計算文本對的得分，適用於文本重排序和語義搜索任務。以下是使用該模型的基本步驟：

1. 安裝Sentence Transformers庫。
2. 加載模型並進行推理。

✨ 主要特性

• 基於Cross Encoder架構，能夠有效計算文本對的相關性得分。
• 微調自microsoft/MiniLM-L12-H384-uncased模型，具有較好的性能。
• 支持最大序列長度為512個標記。
• 輸出單個標籤得分，可用於文本重排序和語義搜索。

📦 安裝指南

首先，你需要安裝Sentence Transformers庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

from sentence_transformers import CrossEncoder

# 從🤗 Hub下載模型
model = CrossEncoder("tomaarsen/reranker-MiniLM-L12-gooaq-bce")
# 獲取文本對的得分
pairs = [
    ['what is the remote desktop connection broker?', 'A remote desktop connection broker is software that allows clients to access various types of server-hosted desktops and applications. ... Load balancing the servers that host the desktops. Managing desktop images. Redirecting multimedia processing to the client.'],
    ['what is the remote desktop connection broker?', 'Remote Desktop Connection (RDC, also called Remote Desktop, formerly Microsoft Terminal Services Client, mstsc or tsclient) is the client application for RDS. It allows a user to remotely log into a networked computer running the terminal services server.'],
    ['what is the remote desktop connection broker?', "['Click the Start menu on your PC and search for Remote Desktop Connection.', 'Launch Remote Desktop Connection and click on Show Options.', 'Select the Local Resources tab and click More.', 'Under Drives, check the box for your C: drive or the drives that contain the files you will transfer and click OK.']"],
    ['what is the remote desktop connection broker?', "['Press the MENU button on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you want to program. ... ', 'Follow the on-screen instructions to finish programming your remote.']"],
    ['what is the remote desktop connection broker?', "['Press MENU on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete the programming.']"],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根據與單個文本的相似度對不同文本進行排序
ranks = model.rank(
    'what is the remote desktop connection broker?',
    [
        'A remote desktop connection broker is software that allows clients to access various types of server-hosted desktops and applications. ... Load balancing the servers that host the desktops. Managing desktop images. Redirecting multimedia processing to the client.',
        'Remote Desktop Connection (RDC, also called Remote Desktop, formerly Microsoft Terminal Services Client, mstsc or tsclient) is the client application for RDS. It allows a user to remotely log into a networked computer running the terminal services server.',
        "['Click the Start menu on your PC and search for Remote Desktop Connection.', 'Launch Remote Desktop Connection and click on Show Options.', 'Select the Local Resources tab and click More.', 'Under Drives, check the box for your C: drive or the drives that contain the files you will transfer and click OK.']",
        "['Press the MENU button on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you want to program. ... ', 'Follow the on-screen instructions to finish programming your remote.']",
        "['Press MENU on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete the programming.']",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers文檔
• 文檔：Cross Encoder文檔
• 倉庫：GitHub上的Sentence Transformers
• Hugging Face：Hugging Face上的Cross Encoders

評估

交叉編碼器重排序（gooaq-dev數據集）

使用CrossEncoderRerankingEvaluator進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": false
}

交叉編碼器重排序（NanoMSMARCO_R100, NanoNFCorpus_R100和NanoNQ_R100數據集）

使用CrossEncoderRerankingEvaluator進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

交叉編碼器Nano BEIR（NanoBEIR_R100_mean數據集）

使用CrossEncoderNanoBEIREvaluator進行評估，參數如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

訓練詳情

訓練數據集

未命名數據集

• 大小：578,402個訓練樣本

• 列：question，answer和label

• 基於前1000個樣本的近似統計信息： | | 問題 | 答案 | 標籤 | |------|------|------|------| | 類型 | 字符串 | 字符串 | 整數 | | 詳情 |

  • 最小長度: 18個字符
  • 平均長度: 42.66個字符
  • 最大長度: 73個字符
  |
  • 最小長度: 51個字符
  • 平均長度: 252.61個字符
  • 最大長度: 368個字符
  |
  • 0: ~82.90%
  • 1: ~17.10%
  |

• 樣本： | 問題 | 答案 | 標籤 | |------|------|------| | what is the remote desktop connection broker? | A remote desktop connection broker is software that allows clients to access various types of server-hosted desktops and applications. ... Load balancing the servers that host the desktops. Managing desktop images. Redirecting multimedia processing to the client. | 1 | | what is the remote desktop connection broker? | Remote Desktop Connection (RDC, also called Remote Desktop, formerly Microsoft Terminal Services Client, mstsc or tsclient) is the client application for RDS. It allows a user to remotely log into a networked computer running the terminal services server. | 0 | | what is the remote desktop connection broker? | ['Click the Start menu on your PC and search for Remote Desktop Connection.', 'Launch Remote Desktop Connection and click on Show Options.', 'Select the Local Resources tab and click More.', 'Under Drives, check the box for your C: drive or the drives that contain the files you will transfer and click OK.'] | 0 |

• 損失函數：使用BinaryCrossEntropyLoss，參數如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 5
}

訓練超參數

非默認超參數

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

所有超參數

訓練日誌

加粗行表示保存的檢查點。

環境影響

使用CodeCarbon測量碳排放：

• 能源消耗：0.143 kWh
• 碳排放：0.056 kg的CO2
• 使用時長：0.391小時

訓練硬件

• 是否使用雲服務：否
• GPU型號：1 x NVIDIA GeForce RTX 3090
• CPU型號：13th Gen Intel(R) Core(TM) i7-13700K
• 內存大小：31.78 GB

框架版本

• Python: 3.11.6
• Sentence Transformers: 3.5.0.dev0
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.1
• Datasets: 3.3.2
• Tokenizers: 0.21.0

🔧 技術細節

本模型基於Cross Encoder架構，通過微調microsoft/MiniLM-L12-H384-uncased模型得到。在訓練過程中，使用了BinaryCrossEntropyLoss作為損失函數，並採用了一系列超參數進行優化。模型的輸入為文本對，輸出為一個得分，表示文本對的相關性。

📄 許可證

本模型使用apache-2.0許可證。

📚 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}