Reranker ModernBERT Base Gooaq Bce

🚀 在GooAQ上训练的ModernBERT-base

这是一个基于answerdotai/ModernBERT-base，使用sentence-transformers库微调得到的交叉编码器模型。它可以为文本对计算得分，这些得分可用于文本重排序和语义搜索。

训练脚本请参考training_gooaq_bce.py。该脚本也在交叉编码器 > 训练概述文档和使用Sentence Transformers v4训练和微调重排序模型博客文章中有所描述。

🚀 快速开始

本模型是一个基于answerdotai/ModernBERT-base微调的交叉编码器模型，可用于文本对得分计算，适用于文本重排序和语义搜索任务。

✨ 主要特性

• 基于预训练的answerdotai/ModernBERT-base模型进行微调，能有效计算文本对的得分。
• 可用于文本重排序和语义搜索，为相关任务提供支持。

📦 安装指南

首先安装Sentence Transformers库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import CrossEncoder

# 从🤗 Hub下载模型
model = CrossEncoder("tomaarsen/reranker-ModernBERT-base-gooaq-bce")
# 获取文本对的得分
pairs = [
    ['why are rye chips so good?', "It makes them taste that much better! The rye chips are tasty because they stand out--they're the saltiest thing in the bag. It's not because rye bread is inherently awesome. ... You could just buy a bag of rye chips."],
    ['why are rye chips so good?', 'There are no substantial technical, nutritional or performance issues associated with rye that would limit its use for pets. Rye is a fairly common ingredient in human foods and beverages. The most prevalent occurrence is in crackers and breads.'],
    ['why are rye chips so good?', 'Bread made wholly from rye flour is made in Germany and called pumpernickel. Rye is unique among grains for having a high level of fibre in its endosperm – not just in its bran. As such, the glycemic index (GI) of rye products is generally lower than products made from wheat and most other grains.'],
    ['why are rye chips so good?', 'KFC Chips – The salt mix on the seasoned chips and the actual chips do not contain any animal products. Our supplier/s of chips and seasoning have confirmed they are suitable for vegans.'],
    ['why are rye chips so good?', 'A study in the American Journal of Clinical Nutrition found that eating rye leads to better blood-sugar control compared to wheat. Rye bread is packed with magnesium, which helps control blood pressure and optimize heart health. Its high levels of soluble fibre can also reduce cholesterol.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'why are rye chips so good?',
    [
        "It makes them taste that much better! The rye chips are tasty because they stand out--they're the saltiest thing in the bag. It's not because rye bread is inherently awesome. ... You could just buy a bag of rye chips.",
        'There are no substantial technical, nutritional or performance issues associated with rye that would limit its use for pets. Rye is a fairly common ingredient in human foods and beverages. The most prevalent occurrence is in crackers and breads.',
        'Bread made wholly from rye flour is made in Germany and called pumpernickel. Rye is unique among grains for having a high level of fibre in its endosperm – not just in its bran. As such, the glycemic index (GI) of rye products is generally lower than products made from wheat and most other grains.',
        'KFC Chips – The salt mix on the seasoned chips and the actual chips do not contain any animal products. Our supplier/s of chips and seasoning have confirmed they are suitable for vegans.',
        'A study in the American Journal of Clinical Nutrition found that eating rye leads to better blood-sugar control compared to wheat. Rye bread is packed with magnesium, which helps control blood pressure and optimize heart health. Its high levels of soluble fibre can also reduce cholesterol.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers文档
• 文档：交叉编码器文档
• 仓库：GitHub上的Sentence Transformers
• Hugging Face：Hugging Face上的交叉编码器

评估

交叉编码器重排序

• 数据集：gooaq-dev
• 评估方法：使用CrossEncoderRerankingEvaluator，参数如下：

  {
      "at_k": 10,
      "always_rerank_positives": false
  }
  

• 数据集：gooaq-dev
• 评估方法：使用CrossEncoderRerankingEvaluator，参数如下：

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

• 数据集：NanoMSMARCO_R100、NanoNFCorpus_R100和NanoNQ_R100
• 评估方法：使用CrossEncoderRerankingEvaluator，参数如下：

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

交叉编码器Nano BEIR

• 数据集：NanoBEIR_R100_mean
• 评估方法：使用CrossEncoderNanoBEIREvaluator，参数如下：

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

训练详情

训练数据集

未命名数据集

• 大小：578,402个训练样本
• 列：question、answer和label
• 基于前1000个样本的近似统计信息： | | 问题 | 答案 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 整数 | | 详情 |
  • 最小：19个字符
  • 平均：45.14个字符
  • 最大：85个字符
  |
  • 最小：65个字符
  • 平均：254.8个字符
  • 最大：379个字符
  |
  • 0：~82.90%
  • 1：~17.10%
  |
• 样本： | 问题 | 答案 | 标签 | |------|------|------| | why are rye chips so good? | It makes them taste that much better! The rye chips are tasty because they stand out--they're the saltiest thing in the bag. It's not because rye bread is inherently awesome. ... You could just buy a bag of rye chips. | 1 | | why are rye chips so good? | There are no substantial technical, nutritional or performance issues associated with rye that would limit its use for pets. Rye is a fairly common ingredient in human foods and beverages. The most prevalent occurrence is in crackers and breads. | 0 | | why are rye chips so good? | Bread made wholly from rye flour is made in Germany and called pumpernickel. Rye is unique among grains for having a high level of fibre in its endosperm – not just in its bran. As such, the glycemic index (GI) of rye products is generally lower than products made from wheat and most other grains. | 0 |
• 损失函数：BinaryCrossEntropyLoss，参数如下：

  {
      "activation_fct": "torch.nn.modules.linear.Identity",
      "pos_weight": 5
  }
  

训练超参数

非默认超参数

• eval_strategy：steps
• per_device_train_batch_size：64
• per_device_eval_batch_size：64
• learning_rate：2e-05
• num_train_epochs：1
• warmup_ratio：0.1
• seed：12
• bf16：True
• dataloader_num_workers：4
• load_best_model_at_end：True

训练日志

注：加粗行表示保存的检查点。

框架版本

• Python：3.11.10
• Sentence Transformers：3.5.0.dev0
• Transformers：4.49.0
• PyTorch：2.5.1+cu124
• Accelerate：1.5.2
• Datasets：2.21.0
• Tokenizers：0.21.0

📄 许可证

本模型使用apache-2.0许可证。

📚 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}