Reranker ModernBERT Base Gooaq Bce

🚀 在GooAQ上训练的ModernBERT-base

这是一个基于answerdotai/ModernBERT-base，使用sentence-transformers库微调得到的交叉编码器模型。它可以计算文本对的得分，可用于文本重排序和语义搜索。

🚀 快速开始

这个模型可以计算文本对的得分，可用于文本重排序和语义搜索。

✨ 主要特性

• 基于answerdotai/ModernBERT-base微调，具有良好的语义理解能力。
• 能够计算文本对的得分，可用于文本重排序和语义搜索。
• 支持最大长度为8192的输入序列。

📦 安装指南

首先安装Sentence Transformers库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import CrossEncoder

# 从🤗 Hub下载模型
model = CrossEncoder("akr2002/reranker-ModernBERT-base-gooaq-bce")
# 获取文本对的得分
pairs = [
    ['how do you find mass?', "Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass."],
    ['how do you find mass?', "In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers."],
    ['how do you find mass?', 'A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV.'],
    ['how do you find mass?', 'Mass is the amount of matter in a body, normally measured in grams or kilograms etc. Weight is a force that pulls on a mass and is measured in Newtons. ... Density basically means how much mass is occupied in a specific volume or space. Different materials of the same size may have different masses because of its density.'],
    ['how do you find mass?', 'Receiver – Mass communication is the transmission of the message to a large number of recipients. This mass of receivers, are often called as mass audience. The Mass audience is large, heterogenous and anonymous in nature. The receivers are scattered across a given village, state or country.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'how do you find mass?',
    [
        "Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass.",
        "In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers.",
        'A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV.',
        'Mass is the amount of matter in a body, normally measured in grams or kilograms etc. Weight is a force that pulls on a mass and is measured in Newtons. ... Density basically means how much mass is occupied in a specific volume or space. Different materials of the same size may have different masses because of its density.',
        'Receiver – Mass communication is the transmission of the message to a large number of recipients. This mass of receivers, are often called as mass audience. The Mass audience is large, heterogenous and anonymous in nature. The receivers are scattered across a given village, state or country.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers文档
• 文档：交叉编码器文档
• 仓库：GitHub上的Sentence Transformers
• Hugging Face：Hugging Face上的交叉编码器

评估

指标

交叉编码器重排序（数据集：gooaq-dev）

使用CrossEncoderRerankingEvaluator进行评估，参数如下：

{
    "at_k": 10,
    "always_rerank_positives": false
}

交叉编码器重排序（数据集：NanoMSMARCO_R100, NanoNFCorpus_R100, NanoNQ_R100）

使用CrossEncoderRerankingEvaluator进行评估，参数如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

交叉编码器Nano BEIR（数据集：NanoBEIR_R100_mean）

使用CrossEncoderNanoBEIREvaluator进行评估，参数如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

训练详情

训练数据集

未命名数据集

• 大小：578,402个训练样本
• 列：question、answer和label
• 基于前1000个样本的近似统计信息： | | 问题 | 答案 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 整数 | | 详情 |
  • 最小：17个字符
  • 平均：44.75个字符
  • 最大：84个字符
  |
  • 最小：54个字符
  • 平均：252.51个字符
  • 最大：388个字符
  |
  • 0：~83.00%
  • 1：~17.00%
  |
• 样本： | 问题 | 答案 | 标签 | |------|------|------| | how do you find mass? | Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass. | 1 | | how do you find mass? | In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers. | 0 | | how do you find mass? | A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV. | 0 |

使用的损失函数为BinaryCrossEntropyLoss，参数如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 5
}

训练超参数

非默认超参数

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

所有超参数

训练日志

加粗行表示保存的检查点。

框架版本

• Python: 3.12.7
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

📄 许可证

本项目采用apache-2.0许可证。

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}