Reranker Msmarco ModernBERT Base Lambdaloss

🚀 基于answerdotai/ModernBERT-base的交叉编码器

本模型是基于answerdotai/ModernBERT-base的交叉编码器，在msmarco数据集上使用sentence-transformers库进行微调。它可以计算文本对的得分，可用于文本重排序和语义搜索。

🚀 快速开始

直接使用（Sentence Transformers）

首先安装Sentence Transformers库：

pip install -U sentence-transformers

然后你可以加载这个模型并进行推理。

from sentence_transformers import CrossEncoder

# 从🤗 Hub下载模型
model = CrossEncoder("tomaarsen/reranker-msmarco-ModernBERT-base-lambdaloss")
# 获取文本对的得分
pairs = [
    ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
    ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
    ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (3,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'How many calories in an egg',
    [
        'There are on average between 55 and 80 calories in an egg depending on its size.',
        'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
        'Most of the calories in an egg come from the yellow yolk in the center.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

✨ 主要特性

• 基于answerdotai/ModernBERT-base模型进行微调，具有良好的文本处理能力。
• 能够计算文本对的得分，可用于文本重排序和语义搜索。
• 支持最大长度为8192个标记的输入序列。

📦 安装指南

安装Sentence Transformers库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import CrossEncoder

# 从🤗 Hub下载模型
model = CrossEncoder("tomaarsen/reranker-msmarco-ModernBERT-base-lambdaloss")
# 获取文本对的得分
pairs = [
    ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
    ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
    ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (3,)

高级用法

from sentence_transformers import CrossEncoder

# 从🤗 Hub下载模型
model = CrossEncoder("tomaarsen/reranker-msmarco-ModernBERT-base-lambdaloss")
# 根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'How many calories in an egg',
    [
        'There are on average between 55 and 80 calories in an egg depending on its size.',
        'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
        'Most of the calories in an egg come from the yellow yolk in the center.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers文档
• 文档：交叉编码器文档
• 仓库：GitHub上的Sentence Transformers
• Hugging Face：Hugging Face上的交叉编码器

评估

指标

交叉编码器重排序

• 数据集：NanoMSMARCO_R100、NanoNFCorpus_R100和NanoNQ_R100
• 使用CrossEncoderRerankingEvaluator进行评估，参数如下：

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

交叉编码器Nano BEIR

• 数据集：NanoBEIR_R100_mean
• 使用CrossEncoderNanoBEIREvaluator进行评估，参数如下：

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

训练详情

训练数据集

• 数据集：msmarco（版本：a0537b6）
• 大小：399,282个训练样本
• 列：query_id、doc_ids和labels

评估数据集

• 数据集：msmarco（版本：a0537b6）
• 大小：1,000个评估样本
• 列：query_id、doc_ids和labels

训练超参数

• 非默认超参数：
  • eval_strategy: steps
  • num_train_epochs: 1
  • warmup_ratio: 0.1
  • seed: 12
  • bf16: True
  • load_best_model_at_end: True

框架版本

• Python: 3.11.10
• Sentence Transformers: 3.5.0.dev0
• Transformers: 4.49.0
• PyTorch: 2.5.1+cu124
• Accelerate: 1.2.0
• Datasets: 2.21.0
• Tokenizers: 0.21.0

🔧 技术细节

损失函数

使用LambdaLoss损失函数，参数如下：

{
    "weighting_scheme": "sentence_transformers.cross_encoder.losses.LambdaLoss.NDCGLoss2PPScheme",
    "k": null,
    "sigma": 1.0,
    "eps": 1e-10,
    "reduction_log": "binary",
    "activation_fct": "torch.nn.modules.linear.Identity",
    "mini_batch_size": 8
}

训练日志

📄 许可证

本模型使用apache-2.0许可证。

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

LambdaLoss

@inproceedings{wang2018lambdaloss,
  title={The lambdaloss framework for ranking metric optimization},
  author={Wang, Xuanhui and Li, Cheng and Golbandi, Nadav and Bendersky, Michael and Najork, Marc},
  booktitle={Proceedings of the 27th ACM international conference on information and knowledge management},
  pages={1313--1322},
  year={2018}
}