Reranker ModernBERT Base Gooaq 1 Epoch 1995000

🚀 基於 answerdotai/ModernBERT-base 的交叉編碼器

這是一個基於 answerdotai/ModernBERT-base 微調的 交叉編碼器 模型，使用 sentence-transformers 庫進行訓練。它可以計算文本對的得分，可用於文本重排序和語義搜索。

✨ 主要特性

• 基於強大的 answerdotai/ModernBERT-base 模型進行微調，繼承了其優秀的語言理解能力。
• 作為交叉編碼器，能夠有效計算文本對之間的相關性得分，適用於文本重排序和語義搜索任務。
• 支持最大長度為 8192 個標記的輸入序列，能夠處理較長的文本。

📦 安裝指南

首先安裝 Sentence Transformers 庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

from sentence_transformers import CrossEncoder

# 從 🤗 Hub 下載模型
model = CrossEncoder("ayushexel/reranker-ModernBERT-base-gooaq-1-epoch-1995000")
# 獲取文本對的得分
pairs = [
    ['is the beryl m762 in pubg mobile?', 'Beryl M762 is a versatile Assault Rifle in PUBG Mobile that has more attachment slots than AKM and uses 7.62 ammunition. The high damage of Beryl M762 makes it a viable option for the players.'],
    ['is the beryl m762 in pubg mobile?', 'The main difference that most people will notice while playing PUBG Mobile Lite after playing on PUBG Mobile is the availability of maps. PUBG Mobile has four maps Erangel, Miramar, Sanhok and Vikendi. PUBG Mobile Lite only has two maps; Erangel in the Classic Mode and War in the Arcade Mode.'],
    ['is the beryl m762 in pubg mobile?', 'PUBG Mobile Lite is the toned-down version of PUBG Mobile, which was developed specifically for players with low-end devices. The game is available for only Android devices at the moment, and there is no way by which you can download it on an iOS device.'],
    ['is the beryl m762 in pubg mobile?', 'Download and play PUBG Mobile on PC with NoxPlayer! PUBG Mobile is a battle royale FPS game developed by Tencent. It is similar to Garena Free Fire and Call of duty Mobile. NoxPlayer is the best emulator to play PUBG Mobile on PC.'],
    ['is the beryl m762 in pubg mobile?', "Can you play PUBG Mobile with a controller? ... For PUBG Mobile, there is no official controller support for the game outside of movement, meaning you can connect a Bluetooth-enabled controller to your mobile device and move around, but the buttons won't have any actions mapped to them."],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根據與單個文本的相似度對不同文本進行排序
ranks = model.rank(
    'is the beryl m762 in pubg mobile?',
    [
        'Beryl M762 is a versatile Assault Rifle in PUBG Mobile that has more attachment slots than AKM and uses 7.62 ammunition. The high damage of Beryl M762 makes it a viable option for the players.',
        'The main difference that most people will notice while playing PUBG Mobile Lite after playing on PUBG Mobile is the availability of maps. PUBG Mobile has four maps Erangel, Miramar, Sanhok and Vikendi. PUBG Mobile Lite only has two maps; Erangel in the Classic Mode and War in the Arcade Mode.',
        'PUBG Mobile Lite is the toned-down version of PUBG Mobile, which was developed specifically for players with low-end devices. The game is available for only Android devices at the moment, and there is no way by which you can download it on an iOS device.',
        'Download and play PUBG Mobile on PC with NoxPlayer! PUBG Mobile is a battle royale FPS game developed by Tencent. It is similar to Garena Free Fire and Call of duty Mobile. NoxPlayer is the best emulator to play PUBG Mobile on PC.',
        "Can you play PUBG Mobile with a controller? ... For PUBG Mobile, there is no official controller support for the game outside of movement, meaning you can connect a Bluetooth-enabled controller to your mobile device and move around, but the buttons won't have any actions mapped to them.",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers 文檔
• 文檔：交叉編碼器文檔
• 倉庫：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的交叉編碼器

評估

指標

交叉編碼器重排序

• 數據集：gooaq-dev
• 使用 CrossEncoderRerankingEvaluator 進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": false
}

交叉編碼器重排序

• 數據集：NanoMSMARCO_R100、NanoNFCorpus_R100 和 NanoNQ_R100
• 使用 CrossEncoderRerankingEvaluator 進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

交叉編碼器 Nano BEIR

• 數據集：NanoBEIR_R100_mean
• 使用 CrossEncoderNanoBEIREvaluator 進行評估，參數如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

訓練詳情

訓練數據集

未命名數據集

• 大小：11,456,701 個訓練樣本
• 列：question、answer 和 label
• 基於前 1000 個樣本的近似統計信息： | | 問題 | 答案 | 標籤 | |--------|-----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|------------------------------------------------| | 類型 | 字符串 | 字符串 | 整數 | | 詳情 |
  • 最小：20 個字符
  • 平均：44.38 個字符
  • 最大：82 個字符
  |
  • 最小：57 個字符
  • 平均：253.74 個字符
  • 最大：358 個字符
  |
  • 0：~82.70%
  • 1：~17.30%
  |
• 樣本： | 問題 | 答案 | 標籤 | |-----------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| | pubg mobile 中有 Beryl M762 嗎？ | Beryl M762 是 PUBG Mobile 中一款多功能突擊步槍，比 AKM 有更多的配件槽，使用 7.62 彈藥。Beryl M762 的高傷害使其成為玩家的可行選擇。 | 1 | | pubg mobile 中有 Beryl M762 嗎？ | 大多數人在玩過 PUBG Mobile 後再玩 PUBG Mobile Lite 時會注意到的主要區別是地圖的可用性。PUBG Mobile 有四張地圖：Erangel、Miramar、Sanhok 和 Vikendi。PUBG Mobile Lite 只有兩張地圖：經典模式中的 Erangel 和街機模式中的 War。 | 0 | | pubg mobile 中有 Beryl M762 嗎？ | PUBG Mobile Lite 是 PUBG Mobile 的簡化版本，專門為低端設備玩家開發。目前該遊戲僅適用於安卓設備，無法在 iOS 設備上下載。 | 0 |
• 損失函數：BinaryCrossEntropyLoss，參數如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 5
}

訓練超參數

非默認超參數

• eval_strategy：按步數評估
• per_device_train_batch_size：256
• per_device_eval_batch_size：256
• learning_rate：2e-05
• num_train_epochs：1
• warmup_ratio：0.1
• seed：12
• bf16：True
• dataloader_num_workers：12
• load_best_model_at_end：True

所有超參數

訓練日誌

框架版本

• Python：3.11.0
• Sentence Transformers：4.0.1
• Transformers：4.50.3
• PyTorch：2.6.0+cu124
• Accelerate：1.5.2
• Datasets：3.5.0
• Tokenizers：0.21.1

📄 許可證

本模型使用 apache-2.0 許可證。

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}