Finetuned Cross Encoder L6 V2

🚀 基于 cross-encoder/ms-marco-MiniLM-L6-v2 的交叉编码器

这是一个基于 cross-encoder/ms-marco-MiniLM-L6-v2 微调的 交叉编码器 模型，使用了 sentence-transformers 库。它可以计算文本对的得分，可用于文本重排序和语义搜索。

🚀 快速开始

本模型可用于计算文本对的得分，适用于文本重排序和语义搜索等任务。下面将为你介绍如何使用该模型。

✨ 主要特性

• 基于 cross-encoder/ms-marco-MiniLM-L6-v2 微调，可计算文本对得分。
• 适用于文本重排序和语义搜索任务。

📦 安装指南

首先，你需要安装 sentence-transformers 库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import CrossEncoder

# 从 Hugging Face Hub 下载模型
model = CrossEncoder("CharlesPing/finetuned-cross-encoder-l6-v2")
# 获取文本对的得分
pairs = [
    ['‘Getting hung up on the exact nature of the records is interesting, and there’s lots of technical work that can be done there, but the main take-home response there is that the trends we’ve been seeing since the 1970s are continuing and have not paused in any way,’ he said.”', 'Rosenzweig also criticized the "waffling—encouraged by the NPOV policy—[which] means that it is hard to discern any overall interpretive stance in Wikipedia history".'],
    ['After the 9/11 terrorist attacks grounded commercial air traffic, "there was a temperature drop while the airplanes weren\'t flying, for the week afterwards."', 'Play media At 9:42\xa0a.m., the Federal Aviation Administration (FAA) grounded all civilian aircraft within the continental U.S., and civilian aircraft already in flight were told to land immediately.'],
    ['But the central message of the IPCC AR4, is confirmed by the peer reviewed literature.', 'Scientific consensus is normally achieved through communication at conferences, publication in the scientific literature, replication (reproducible results by others), and peer review.'],
    ['"Many people think the science of climate change is settled.', 'During his administration, the bridge from Filadelfia and Liberia was constructed, as was the Old National Theater.'],
    ['“Even if you could calculate some sort of meaningful global temperature statistic, the figure would be unimportant.', 'Quantitative information or data is based on quantities obtained using a quantifiable measurement process.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    '‘Getting hung up on the exact nature of the records is interesting, and there’s lots of technical work that can be done there, but the main take-home response there is that the trends we’ve been seeing since the 1970s are continuing and have not paused in any way,’ he said.”',
    [
        'Rosenzweig also criticized the "waffling—encouraged by the NPOV policy—[which] means that it is hard to discern any overall interpretive stance in Wikipedia history".',
        'Play media At 9:42\xa0a.m., the Federal Aviation Administration (FAA) grounded all civilian aircraft within the continental U.S., and civilian aircraft already in flight were told to land immediately.',
        'Scientific consensus is normally achieved through communication at conferences, publication in the scientific literature, replication (reproducible results by others), and peer review.',
        'During his administration, the bridge from Filadelfia and Liberia was constructed, as was the Old National Theater.',
        'Quantitative information or data is based on quantities obtained using a quantifiable measurement process.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers 文档
• 文档：交叉编码器文档
• 仓库：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的交叉编码器

评估

指标

交叉编码器重排序

• 数据集：cross-rerank-dev-mixed-neg
• 使用 CrossEncoderRerankingEvaluator 进行评估，参数如下：

  {
      "at_k": 10
  }
  

训练详情

训练数据集

未命名数据集

• 大小：22,258 个训练样本
• 列：sentence_0、sentence_1 和 label
• 基于前 1000 个样本的近似统计信息： | | sentence_0 | sentence_1 | label | |------|------|------|------| | 类型 | 字符串 | 字符串 | 浮点数 | | 详情 |
  • 最小长度：26 个字符
  • 平均长度：121.91 个字符
  • 最大长度：319 个字符
  |
  • 最小长度：36 个字符
  • 平均长度：140.85 个字符
  • 最大长度：573 个字符
  |
  • 最小值：0.0
  • 平均值：0.16
  • 最大值：1.0
  |
• 样本： | sentence_0 | sentence_1 | label | |------|------|------| | ‘Getting hung up on the exact nature of the records is interesting, and there’s lots of technical work that can be done there, but the main take-home response there is that the trends we’ve been seeing since the 1970s are continuing and have not paused in any way,’ he said.” | Rosenzweig also criticized the "waffling—encouraged by the NPOV policy—[which] means that it is hard to discern any overall interpretive stance in Wikipedia history". | 1.0 | | After the 9/11 terrorist attacks grounded commercial air traffic, "there was a temperature drop while the airplanes weren't flying, for the week afterwards." | Play media At 9:42 a.m., the Federal Aviation Administration (FAA) grounded all civilian aircraft within the continental U.S., and civilian aircraft already in flight were told to land immediately. | 1.0 | | But the central message of the IPCC AR4, is confirmed by the peer reviewed literature. | Scientific consensus is normally achieved through communication at conferences, publication in the scientific literature, replication (reproducible results by others), and peer review. | 1.0 |
• 损失函数：FitMixinLoss

训练超参数

非默认超参数

• eval_strategy：按步数评估
• per_device_train_batch_size：16
• per_device_eval_batch_size：16

所有超参数

训练日志

框架版本

• Python: 3.11.12
• Sentence Transformers: 4.1.0
• Transformers: 4.51.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.5.1
• Tokenizers: 0.21.1

🔧 技术细节

本模型基于 cross-encoder/ms-marco-MiniLM-L6-v2 微调，使用 sentence-transformers 库进行训练。训练过程中使用了特定的超参数和损失函数，以优化模型在文本重排序任务上的性能。

📄 许可证

文档中未提及相关许可证信息。

📖 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}