Reranker ModernBERT Base Gooaq 1 Epoch 1995000

🚀 基于 answerdotai/ModernBERT-base 的交叉编码器

这是一个基于 answerdotai/ModernBERT-base 微调的 交叉编码器 模型，使用 sentence-transformers 库进行训练。它可以计算文本对的得分，可用于文本重排序和语义搜索。

✨ 主要特性

• 基于强大的 answerdotai/ModernBERT-base 模型进行微调，继承了其优秀的语言理解能力。
• 作为交叉编码器，能够有效计算文本对之间的相关性得分，适用于文本重排序和语义搜索任务。
• 支持最大长度为 8192 个标记的输入序列，能够处理较长的文本。

📦 安装指南

首先安装 Sentence Transformers 库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import CrossEncoder

# 从 🤗 Hub 下载模型
model = CrossEncoder("ayushexel/reranker-ModernBERT-base-gooaq-1-epoch-1995000")
# 获取文本对的得分
pairs = [
    ['is the beryl m762 in pubg mobile?', 'Beryl M762 is a versatile Assault Rifle in PUBG Mobile that has more attachment slots than AKM and uses 7.62 ammunition. The high damage of Beryl M762 makes it a viable option for the players.'],
    ['is the beryl m762 in pubg mobile?', 'The main difference that most people will notice while playing PUBG Mobile Lite after playing on PUBG Mobile is the availability of maps. PUBG Mobile has four maps Erangel, Miramar, Sanhok and Vikendi. PUBG Mobile Lite only has two maps; Erangel in the Classic Mode and War in the Arcade Mode.'],
    ['is the beryl m762 in pubg mobile?', 'PUBG Mobile Lite is the toned-down version of PUBG Mobile, which was developed specifically for players with low-end devices. The game is available for only Android devices at the moment, and there is no way by which you can download it on an iOS device.'],
    ['is the beryl m762 in pubg mobile?', 'Download and play PUBG Mobile on PC with NoxPlayer! PUBG Mobile is a battle royale FPS game developed by Tencent. It is similar to Garena Free Fire and Call of duty Mobile. NoxPlayer is the best emulator to play PUBG Mobile on PC.'],
    ['is the beryl m762 in pubg mobile?', "Can you play PUBG Mobile with a controller? ... For PUBG Mobile, there is no official controller support for the game outside of movement, meaning you can connect a Bluetooth-enabled controller to your mobile device and move around, but the buttons won't have any actions mapped to them."],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'is the beryl m762 in pubg mobile?',
    [
        'Beryl M762 is a versatile Assault Rifle in PUBG Mobile that has more attachment slots than AKM and uses 7.62 ammunition. The high damage of Beryl M762 makes it a viable option for the players.',
        'The main difference that most people will notice while playing PUBG Mobile Lite after playing on PUBG Mobile is the availability of maps. PUBG Mobile has four maps Erangel, Miramar, Sanhok and Vikendi. PUBG Mobile Lite only has two maps; Erangel in the Classic Mode and War in the Arcade Mode.',
        'PUBG Mobile Lite is the toned-down version of PUBG Mobile, which was developed specifically for players with low-end devices. The game is available for only Android devices at the moment, and there is no way by which you can download it on an iOS device.',
        'Download and play PUBG Mobile on PC with NoxPlayer! PUBG Mobile is a battle royale FPS game developed by Tencent. It is similar to Garena Free Fire and Call of duty Mobile. NoxPlayer is the best emulator to play PUBG Mobile on PC.',
        "Can you play PUBG Mobile with a controller? ... For PUBG Mobile, there is no official controller support for the game outside of movement, meaning you can connect a Bluetooth-enabled controller to your mobile device and move around, but the buttons won't have any actions mapped to them.",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers 文档
• 文档：交叉编码器文档
• 仓库：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的交叉编码器

评估

指标

交叉编码器重排序

• 数据集：gooaq-dev
• 使用 CrossEncoderRerankingEvaluator 进行评估，参数如下：

{
    "at_k": 10,
    "always_rerank_positives": false
}

交叉编码器重排序

• 数据集：NanoMSMARCO_R100、NanoNFCorpus_R100 和 NanoNQ_R100
• 使用 CrossEncoderRerankingEvaluator 进行评估，参数如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

交叉编码器 Nano BEIR

• 数据集：NanoBEIR_R100_mean
• 使用 CrossEncoderNanoBEIREvaluator 进行评估，参数如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

训练详情

训练数据集

未命名数据集

• 大小：11,456,701 个训练样本
• 列：question、answer 和 label
• 基于前 1000 个样本的近似统计信息： | | 问题 | 答案 | 标签 | |--------|-----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|------------------------------------------------| | 类型 | 字符串 | 字符串 | 整数 | | 详情 |
  • 最小：20 个字符
  • 平均：44.38 个字符
  • 最大：82 个字符
  |
  • 最小：57 个字符
  • 平均：253.74 个字符
  • 最大：358 个字符
  |
  • 0：~82.70%
  • 1：~17.30%
  |
• 样本： | 问题 | 答案 | 标签 | |-----------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| | pubg mobile 中有 Beryl M762 吗？ | Beryl M762 是 PUBG Mobile 中一款多功能突击步枪，比 AKM 有更多的配件槽，使用 7.62 弹药。Beryl M762 的高伤害使其成为玩家的可行选择。 | 1 | | pubg mobile 中有 Beryl M762 吗？ | 大多数人在玩过 PUBG Mobile 后再玩 PUBG Mobile Lite 时会注意到的主要区别是地图的可用性。PUBG Mobile 有四张地图：Erangel、Miramar、Sanhok 和 Vikendi。PUBG Mobile Lite 只有两张地图：经典模式中的 Erangel 和街机模式中的 War。 | 0 | | pubg mobile 中有 Beryl M762 吗？ | PUBG Mobile Lite 是 PUBG Mobile 的简化版本，专门为低端设备玩家开发。目前该游戏仅适用于安卓设备，无法在 iOS 设备上下载。 | 0 |
• 损失函数：BinaryCrossEntropyLoss，参数如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 5
}

训练超参数

非默认超参数

• eval_strategy：按步数评估
• per_device_train_batch_size：256
• per_device_eval_batch_size：256
• learning_rate：2e-05
• num_train_epochs：1
• warmup_ratio：0.1
• seed：12
• bf16：True
• dataloader_num_workers：12
• load_best_model_at_end：True

所有超参数

训练日志

框架版本

• Python：3.11.0
• Sentence Transformers：4.0.1
• Transformers：4.50.3
• PyTorch：2.6.0+cu124
• Accelerate：1.5.2
• Datasets：3.5.0
• Tokenizers：0.21.1

📄 许可证

本模型使用 apache-2.0 许可证。

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}