Reranker Gte Multilingual Base Msmarco Bce Ep 2

🚀 CrossEncoder

CrossEncoder 是一个基于 sentence-transformers 库，在 msmarco 数据集上训练的 Cross Encoder 模型。它可以计算文本对的得分，可用于文本重排序和语义搜索。

✨ 主要特性

• 能够计算文本对的得分，适用于文本重排序和语义搜索任务。
• 支持长序列输入，最大序列长度可达 8192 个标记。

📦 安装指南

首先，安装 Sentence Transformers 库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import CrossEncoder

# 从 🤗 Hub 下载模型
model = CrossEncoder("skfrost19/reranker-gte-multilingual-base-msmarco-bce-ep-2")
# 获取文本对的得分
pairs = [
    ['what symptoms might a patient with a tmd have', 'TMD sufferers have a long list of symptoms, including chronic pain (https://youtu.be/SvMaJb8o2RI), many of which are in common with Parkinsonâ\x80\x99s disease (PD) symptoms.'],
    ['what is a thermal protector', 'The word hero comes from the Greek á¼¥Ï\x81Ï\x89Ï\x82 (hÄ\x93rÅ\x8ds), hero, warrior, particularly one such as Heracles with divine ancestry or later given divine honors. literally protector or defender.'],
    ['how many copies of call of duty wwii sold', 'Call of Duty 3. Call of Duty 3 is a World War II first-person shooter and the third installment in the Call of Duty video game series. Released on November 7, 2006, the game was developed by Treyarch, and was the first major installment in the Call of Duty series not to be developed by Infinity Ward. It was also the first not to be released on the PC platform. It was released on the PlayStation 2, PlayStation 3, Wii, Xbox, and Xbox 360.'],
    ['what is the desired temperature for the fresh food compartment in a refrigerator', 'A refrigerator maintains a temperature a few degrees above the freezing point of water. Optimum temperature range for perishable food storage is 3 to 5 °C (37 to 41 °F).emperature settings for refrigerator and freezer compartments are often given arbitrary numbers by manufacturers (for example, 1 through 9, warmest to coldest), but generally 3 to 5 °C (37 to 41 °F) is ideal for the refrigerator compartment and â\x88\x9218 °C (0 °F) for the freezer.'],
    ['what is gsm alarm system', 'Iâ\x80\x99m sure you would have these questions in your mind when you heard GSM alarm system at the first time. GSM alarm system is an alarm system that operating through GSM (global system for mobile communications) network; not requiring a telephone line.urthermore, in the case of burglar entering the premises and cutting the telephone line, the GSM alarm would not be affected and still work as it does not require the use of a fixed phone line. So this security alarm is ideal for the place where no fixed phone line or hard to get one.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根据与单个文本的相似度对不同文本进行排序
ranks = model.rank(
    'what symptoms might a patient with a tmd have',
    [
        'TMD sufferers have a long list of symptoms, including chronic pain (https://youtu.be/SvMaJb8o2RI), many of which are in common with Parkinsonâ\x80\x99s disease (PD) symptoms.',
        'The word hero comes from the Greek á¼¥Ï\x81Ï\x89Ï\x82 (hÄ\x93rÅ\x8ds), hero, warrior, particularly one such as Heracles with divine ancestry or later given divine honors. literally protector or defender.',
        'Call of Duty 3. Call of Duty 3 is a World War II first-person shooter and the third installment in the Call of Duty video game series. Released on November 7, 2006, the game was developed by Treyarch, and was the first major installment in the Call of Duty series not to be developed by Infinity Ward. It was also the first not to be released on the PC platform. It was released on the PlayStation 2, PlayStation 3, Wii, Xbox, and Xbox 360.',
        'A refrigerator maintains a temperature a few degrees above the freezing point of water. Optimum temperature range for perishable food storage is 3 to 5 °C (37 to 41 °F).emperature settings for refrigerator and freezer compartments are often given arbitrary numbers by manufacturers (for example, 1 through 9, warmest to coldest), but generally 3 to 5 °C (37 to 41 °F) is ideal for the refrigerator compartment and â\x88\x9218 °C (0 °F) for the freezer.',
        'Iâ\x80\x99m sure you would have these questions in your mind when you heard GSM alarm system at the first time. GSM alarm system is an alarm system that operating through GSM (global system for mobile communications) network; not requiring a telephone line.urthermore, in the case of burglar entering the premises and cutting the telephone line, the GSM alarm would not be affected and still work as it does not require the use of a fixed phone line. So this security alarm is ideal for the place where no fixed phone line or hard to get one.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers 文档
• 文档：Cross Encoder 文档
• 仓库：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的 Cross Encoders

评估

评估指标

Cross Encoder 重排序

• 数据集：NanoMSMARCO_R100、NanoNFCorpus_R100 和 NanoNQ_R100
• 评估方法：使用 CrossEncoderRerankingEvaluator 进行评估，参数如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

Cross Encoder Nano BEIR

• 数据集：NanoBEIR_R100_mean
• 评估方法：使用 CrossEncoderNanoBEIREvaluator 进行评估，参数如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

训练详情

训练数据集

• 数据集：msmarco，版本为 9e329ed
• 大小：1,990,000 个训练样本
• 列：query、passage 和 score
• 近似统计信息（基于前 1000 个样本）： | | 查询 | 段落 | 得分 | | ---- | ---- | ---- | ---- | | 类型 | 字符串 | 字符串 | 浮点数 | | 详情 |
  • 最小长度：11 个字符
  • 平均长度：34.61 个字符
  • 最大长度：124 个字符
  |
  • 最小长度：82 个字符
  • 平均长度：357.43 个字符
  • 最大长度：1034 个字符
  |
  • 最小值：0.0
  • 平均值：0.49
  • 最大值：1.0
  |
• 样本： | 查询 | 段落 | 得分 | | ---- | ---- | ---- | | what causes your tailbone to hurt | A coccyx injury results in pain and discomfort in the tailbone area (the condition is called coccydynia). These injuries may result in a bruise, dislocation, or fracture (break) of the coccyx. Although they may be slow to heal, the majority of coccyx injuries can be managed with cautious treatment.ost tailbone injuries are caused by trauma to the coccyx area. 1 A fall onto the tailbone in the seated position, usually against a hard surface, is the most common cause of coccyx injuries. 2 A direct blow to the tailbone, such as those that occur during contact sports, can injure the coccyx. | 1.0 | | what muscles do trunk lateral flexion | It’s the same with the External Obliques, but unlike the External Obliques, they are not visible when fully developed. Action: 1 Supports abdominal wall, assists forced respiration, aids raising intra-abdominal pressure and, with muscles of other side, abducts and rotates trunk. 2 Contraction of one side alone laterally bends the trunk to that side and rotates the trunk to the other side. | 0.0 | | brake horsepower definition | When the brake lights will not come on, the first thing to check is the third-brake light. If it too is not working, the brake-light switch, a bad fuse or an unplugged harness is likely.ull up on the brake pedal and if the lights go out, switch mis-alignment or pedal position error is the likely cause. The final possibility is a wire shorted to power. Unplug the brake-light switch and if the lights stay on, a short circuit is the case. | 0.0 |
• 损失函数：BinaryCrossEntropyLoss，参数如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": null
}

评估数据集

• 数据集：msmarco，版本为 9e329ed
• 大小：10,000 个评估样本
• 列：query、passage 和 score
• 近似统计信息（基于前 1000 个样本）： | | 查询 | 段落 | 得分 | | ---- | ---- | ---- | ---- | | 类型 | 字符串 | 字符串 | 浮点数 | | 详情 |
  • 最小长度：9 个字符
  • 平均长度：33.72 个字符
  • 最大长度：193 个字符
  |
  • 最小长度：55 个字符
  • 平均长度：353.35 个字符
  • 最大长度：895 个字符
  |
  • 最小值：0.0
  • 平均值：0.5
  • 最大值：1.0
  |
• 样本： | 查询 | 段落 | 得分 | | ---- | ---- | ---- | | what symptoms might a patient with a tmd have | TMD sufferers have a long list of symptoms, including chronic pain (https://youtu.be/SvMaJb8o2RI), many of which are in common with Parkinson’s disease (PD) symptoms. | 1.0 | | what is a thermal protector | The word hero comes from the Greek ἥρως (hērōs), hero, warrior, particularly one such as Heracles with divine ancestry or later given divine honors. literally protector or defender. | 0.0 | | how many copies of call of duty wwii sold | Call of Duty 3. Call of Duty 3 is a World War II first-person shooter and the third installment in the Call of Duty video game series. Released on November 7, 2006, the game was developed by Treyarch, and was the first major installment in the Call of Duty series not to be developed by Infinity Ward. It was also the first not to be released on the PC platform. It was released on the PlayStation 2, PlayStation 3, Wii, Xbox, and Xbox 360. | 0.0 |
• 损失函数：BinaryCrossEntropyLoss，参数如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": null
}

训练超参数

非默认超参数

• eval_strategy: steps
• per_device_train_batch_size: 156
• per_device_eval_batch_size: 156
• learning_rate: 2e-05
• num_train_epochs: 2
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

所有超参数

训练日志

注：加粗行表示保存的检查点。

框架版本

• Python: 3.11.5
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.5.0
• Tokenizers: 0.21.1

📄 许可证

文档中未提及该模型的许可证信息。

📚 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}