Finetuned Ce Climate Multineg V1

🚀 基于 cross-encoder/ms-marco-MiniLM-L12-v2 的交叉编码器

这是一个基于 Cross Encoder 的模型，它在 climate-cross-encoder-mixed-neg-v3 数据集上，使用 sentence-transformers 库对 cross-encoder/ms-marco-MiniLM-L12-v2 进行微调得到。该模型可以为文本对计算分数，可用于文本重排序和语义搜索。

🚀 快速开始

本模型可用于计算文本对的分数，进而实现文本重排序和语义搜索。下面将详细介绍如何使用该模型。

✨ 主要特性

• 基于 cross-encoder/ms-marco-MiniLM-L12-v2 进行微调，在 climate-cross-encoder-mixed-neg-v3 数据集上训练。
• 能够计算文本对的分数，用于文本重排序和语义搜索。
• 支持使用 sentence-transformers 库进行推理和微调。

📦 安装指南

首先，你需要安装 sentence-transformers 库：

pip install -U sentence-transformers

💻 使用示例

基础用法

以下是如何加载模型并进行推理的示例：

from sentence_transformers import CrossEncoder

# 从 Hugging Face Hub 下载模型
model = CrossEncoder("CharlesPing/finetuned-ce-climate-multineg-v1")
# 获取文本对的分数
pairs = [
    ['Scientific analysis of past climates\xa0shows that greenhouse gasses, principally CO2,\xa0have controlled most ancient\xa0climate changes.', 'Greenhouse gases, in particular carbon dioxide and methane, played a significant role during the Eocene in controlling the surface temperature.'],
    ['Scientific analysis of past climates\xa0shows that greenhouse gasses, principally CO2,\xa0have controlled most ancient\xa0climate changes.', 'Climatic geomorphology is of limited use to study recent (Quaternary, Holocene) large climate changes since there are seldom discernible in the geomorphological record.'],
    ['Scientific analysis of past climates\xa0shows that greenhouse gasses, principally CO2,\xa0have controlled most ancient\xa0climate changes.', 'There is also a close correlation between CO2 and temperature, where CO2 has a strong control over global temperatures in Earth history.'],
    ['Scientific analysis of past climates\xa0shows that greenhouse gasses, principally CO2,\xa0have controlled most ancient\xa0climate changes.', 'While scientists knew of past climate change such as the ice ages, the concept of climate as unchanging was useful in the development of a general theory of what determines climate.'],
    ['Scientific analysis of past climates\xa0shows that greenhouse gasses, principally CO2,\xa0have controlled most ancient\xa0climate changes.', 'Some long term modifications along the history of the planet have been significant, such as the incorporation of oxygen to the atmosphere.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'Scientific analysis of past climates\xa0shows that greenhouse gasses, principally CO2,\xa0have controlled most ancient\xa0climate changes.',
    [
        'Greenhouse gases, in particular carbon dioxide and methane, played a significant role during the Eocene in controlling the surface temperature.',
        'Climatic geomorphology is of limited use to study recent (Quaternary, Holocene) large climate changes since there are seldom discernible in the geomorphological record.',
        'There is also a close correlation between CO2 and temperature, where CO2 has a strong control over global temperatures in Earth history.',
        'While scientists knew of past climate change such as the ice ages, the concept of climate as unchanging was useful in the development of a general theory of what determines climate.',
        'Some long term modifications along the history of the planet have been significant, such as the incorporation of oxygen to the atmosphere.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型资源

• 文档：Sentence Transformers 文档
• 文档：Cross Encoder 文档
• 代码仓库：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的交叉编码器

评估

指标

交叉编码器重排序

• 数据集：climate-rerank-multineg
• 使用 CrossEncoderRerankingEvaluator 进行评估，参数如下：

  {
      "at_k": 1,
      "always_rerank_positives": false
  }
  

训练详情

训练数据集

climate-cross-encoder-mixed-neg-v3

• 数据集：climate-cross-encoder-mixed-neg-v3，版本为 cd49b57
• 大小：41,052 个训练样本
• 列：query、doc 和 label
• 基于前 1000 个样本的近似统计信息：

  	查询	文档	标签
  类型	字符串	字符串	浮点数
  详情	最小长度：49 个字符 平均长度：140.03 个字符 最大长度：306 个字符	最小长度：4 个字符 平均长度：136.03 个字符 最大长度：731 个字符	最小值：0.0 平均值：0.09 最大值：1.0

• 样本：

  查询	文档	标签
  “A leading Canadian authority on polar bears, Mitch Taylor, said: ‘We’re seeing an increase in bears that’s really unprecedented, and in places where we’re seeing a decrease in the population	Warnings about the future of the polar bear are often contrasted with the fact that worldwide population estimates have increased over the past 50 years and are relatively stable today.	1.0
  “A leading Canadian authority on polar bears, Mitch Taylor, said: ‘We’re seeing an increase in bears that’s really unprecedented, and in places where we’re seeing a decrease in the population	Species distribution models of recent years indicate that the deer tick, known as "I. scapularis," is pushing its distribution to higher latitudes of the Northeastern United States and Canada, as well as pushing and maintaining populations in the South Central and Northern Midwest regions of the United States.	0.0
  “A leading Canadian authority on polar bears, Mitch Taylor, said: ‘We’re seeing an increase in bears that’s really unprecedented, and in places where we’re seeing a decrease in the population	Bear and deer are among the animals present.	0.0

• 损失函数：BinaryCrossEntropyLoss，参数如下：

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

评估数据集

climate-cross-encoder-mixed-neg-v3

• 数据集：climate-cross-encoder-mixed-neg-v3，版本为 cd49b57
• 大小：4,290 个评估样本
• 列：query、doc 和 label
• 基于前 1000 个样本的近似统计信息：

  	查询	文档	标签
  类型	字符串	字符串	浮点数
  详情	最小长度：39 个字符 平均长度：116.67 个字符 最大长度：240 个字符	最小长度：18 个字符 平均长度：132.92 个字符 最大长度：731 个字符	最小值：0.0 平均值：0.09 最大值：1.0

• 样本：

  查询	文档	标签
  Scientific analysis of past climates shows that greenhouse gasses, principally CO2, have controlled most ancient climate changes.	Greenhouse gases, in particular carbon dioxide and methane, played a significant role during the Eocene in controlling the surface temperature.	1.0
  Scientific analysis of past climates shows that greenhouse gasses, principally CO2, have controlled most ancient climate changes.	Climatic geomorphology is of limited use to study recent (Quaternary, Holocene) large climate changes since there are seldom discernible in the geomorphological record.	0.0
  Scientific analysis of past climates shows that greenhouse gasses, principally CO2, have controlled most ancient climate changes.	There is also a close correlation between CO2 and temperature, where CO2 has a strong control over global temperatures in Earth history.	0.0

• 损失函数：BinaryCrossEntropyLoss，参数如下：

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

训练超参数

非默认超参数

• eval_strategy：steps
• per_device_train_batch_size：16
• per_device_eval_batch_size：32
• learning_rate：2e-05
• warmup_ratio：0.1
• fp16：True
• load_best_model_at_end：True

所有超参数

训练日志

• 加粗行表示保存的检查点。

框架版本

• Python：3.11.12
• Sentence Transformers：4.1.0
• Transformers：4.51.3
• PyTorch：2.6.0+cu124
• Accelerate：1.6.0
• Datasets：3.6.0
• Tokenizers：0.21.1

📄 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}