Reranker MiniLM L12 Gooaq Bce

🚀 MiniLM-L12-H384-uncased在GooAQ上训练的模型

这是一个基于Cross Encoder的模型，它使用sentence-transformers库从microsoft/MiniLM-L12-H384-uncased微调而来。该模型可以计算文本对的得分，可用于文本重排序和语义搜索。

🚀 快速开始

本模型可用于计算文本对的得分，适用于文本重排序和语义搜索任务。以下是使用该模型的基本步骤：

1. 安装Sentence Transformers库。
2. 加载模型并进行推理。

✨ 主要特性

• 基于Cross Encoder架构，能够有效计算文本对的相关性得分。
• 微调自microsoft/MiniLM-L12-H384-uncased模型，具有较好的性能。
• 支持最大序列长度为512个标记。
• 输出单个标签得分，可用于文本重排序和语义搜索。

📦 安装指南

首先，你需要安装Sentence Transformers库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import CrossEncoder

# 从🤗 Hub下载模型
model = CrossEncoder("tomaarsen/reranker-MiniLM-L12-gooaq-bce")
# 获取文本对的得分
pairs = [
    ['what is the remote desktop connection broker?', 'A remote desktop connection broker is software that allows clients to access various types of server-hosted desktops and applications. ... Load balancing the servers that host the desktops. Managing desktop images. Redirecting multimedia processing to the client.'],
    ['what is the remote desktop connection broker?', 'Remote Desktop Connection (RDC, also called Remote Desktop, formerly Microsoft Terminal Services Client, mstsc or tsclient) is the client application for RDS. It allows a user to remotely log into a networked computer running the terminal services server.'],
    ['what is the remote desktop connection broker?', "['Click the Start menu on your PC and search for Remote Desktop Connection.', 'Launch Remote Desktop Connection and click on Show Options.', 'Select the Local Resources tab and click More.', 'Under Drives, check the box for your C: drive or the drives that contain the files you will transfer and click OK.']"],
    ['what is the remote desktop connection broker?', "['Press the MENU button on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you want to program. ... ', 'Follow the on-screen instructions to finish programming your remote.']"],
    ['what is the remote desktop connection broker?', "['Press MENU on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete the programming.']"],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'what is the remote desktop connection broker?',
    [
        'A remote desktop connection broker is software that allows clients to access various types of server-hosted desktops and applications. ... Load balancing the servers that host the desktops. Managing desktop images. Redirecting multimedia processing to the client.',
        'Remote Desktop Connection (RDC, also called Remote Desktop, formerly Microsoft Terminal Services Client, mstsc or tsclient) is the client application for RDS. It allows a user to remotely log into a networked computer running the terminal services server.',
        "['Click the Start menu on your PC and search for Remote Desktop Connection.', 'Launch Remote Desktop Connection and click on Show Options.', 'Select the Local Resources tab and click More.', 'Under Drives, check the box for your C: drive or the drives that contain the files you will transfer and click OK.']",
        "['Press the MENU button on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you want to program. ... ', 'Follow the on-screen instructions to finish programming your remote.']",
        "['Press MENU on your remote.', 'Select Parental Favs & Setup > System Setup > Remote or Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete the programming.']",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers文档
• 文档：Cross Encoder文档
• 仓库：GitHub上的Sentence Transformers
• Hugging Face：Hugging Face上的Cross Encoders

评估

交叉编码器重排序（gooaq-dev数据集）

使用CrossEncoderRerankingEvaluator进行评估，参数如下：

{
    "at_k": 10,
    "always_rerank_positives": false
}

交叉编码器重排序（NanoMSMARCO_R100, NanoNFCorpus_R100和NanoNQ_R100数据集）

使用CrossEncoderRerankingEvaluator进行评估，参数如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

交叉编码器Nano BEIR（NanoBEIR_R100_mean数据集）

使用CrossEncoderNanoBEIREvaluator进行评估，参数如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

训练详情

训练数据集

未命名数据集

• 大小：578,402个训练样本

• 列：question，answer和label

• 基于前1000个样本的近似统计信息： | | 问题 | 答案 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 整数 | | 详情 |

  • 最小长度: 18个字符
  • 平均长度: 42.66个字符
  • 最大长度: 73个字符
  |
  • 最小长度: 51个字符
  • 平均长度: 252.61个字符
  • 最大长度: 368个字符
  |
  • 0: ~82.90%
  • 1: ~17.10%
  |

• 样本： | 问题 | 答案 | 标签 | |------|------|------| | what is the remote desktop connection broker? | A remote desktop connection broker is software that allows clients to access various types of server-hosted desktops and applications. ... Load balancing the servers that host the desktops. Managing desktop images. Redirecting multimedia processing to the client. | 1 | | what is the remote desktop connection broker? | Remote Desktop Connection (RDC, also called Remote Desktop, formerly Microsoft Terminal Services Client, mstsc or tsclient) is the client application for RDS. It allows a user to remotely log into a networked computer running the terminal services server. | 0 | | what is the remote desktop connection broker? | ['Click the Start menu on your PC and search for Remote Desktop Connection.', 'Launch Remote Desktop Connection and click on Show Options.', 'Select the Local Resources tab and click More.', 'Under Drives, check the box for your C: drive or the drives that contain the files you will transfer and click OK.'] | 0 |

• 损失函数：使用BinaryCrossEntropyLoss，参数如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 5
}

训练超参数

非默认超参数

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

所有超参数

训练日志

加粗行表示保存的检查点。

环境影响

使用CodeCarbon测量碳排放：

• 能源消耗：0.143 kWh
• 碳排放：0.056 kg的CO2
• 使用时长：0.391小时

训练硬件

• 是否使用云服务：否
• GPU型号：1 x NVIDIA GeForce RTX 3090
• CPU型号：13th Gen Intel(R) Core(TM) i7-13700K
• 内存大小：31.78 GB

框架版本

• Python: 3.11.6
• Sentence Transformers: 3.5.0.dev0
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.1
• Datasets: 3.3.2
• Tokenizers: 0.21.0

🔧 技术细节

本模型基于Cross Encoder架构，通过微调microsoft/MiniLM-L12-H384-uncased模型得到。在训练过程中，使用了BinaryCrossEntropyLoss作为损失函数，并采用了一系列超参数进行优化。模型的输入为文本对，输出为一个得分，表示文本对的相关性。

📄 许可证

本模型使用apache-2.0许可证。

📚 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}