Reranker ModernBERT Base Gooaq Bce

🚀 在GooAQ上訓練的ModernBERT-base

這是一個基於answerdotai/ModernBERT-base，使用sentence-transformers庫微調得到的交叉編碼器模型。它可以計算文本對的得分，可用於文本重排序和語義搜索。

🚀 快速開始

這個模型可以計算文本對的得分，可用於文本重排序和語義搜索。

✨ 主要特性

• 基於answerdotai/ModernBERT-base微調，具有良好的語義理解能力。
• 能夠計算文本對的得分，可用於文本重排序和語義搜索。
• 支持最大長度為8192的輸入序列。

📦 安裝指南

首先安裝Sentence Transformers庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

from sentence_transformers import CrossEncoder

# 從🤗 Hub下載模型
model = CrossEncoder("akr2002/reranker-ModernBERT-base-gooaq-bce")
# 獲取文本對的得分
pairs = [
    ['how do you find mass?', "Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass."],
    ['how do you find mass?', "In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers."],
    ['how do you find mass?', 'A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV.'],
    ['how do you find mass?', 'Mass is the amount of matter in a body, normally measured in grams or kilograms etc. Weight is a force that pulls on a mass and is measured in Newtons. ... Density basically means how much mass is occupied in a specific volume or space. Different materials of the same size may have different masses because of its density.'],
    ['how do you find mass?', 'Receiver – Mass communication is the transmission of the message to a large number of recipients. This mass of receivers, are often called as mass audience. The Mass audience is large, heterogenous and anonymous in nature. The receivers are scattered across a given village, state or country.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 或者根據與單個文本的相似度對不同文本進行排序
ranks = model.rank(
    'how do you find mass?',
    [
        "Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass.",
        "In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers.",
        'A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV.',
        'Mass is the amount of matter in a body, normally measured in grams or kilograms etc. Weight is a force that pulls on a mass and is measured in Newtons. ... Density basically means how much mass is occupied in a specific volume or space. Different materials of the same size may have different masses because of its density.',
        'Receiver – Mass communication is the transmission of the message to a large number of recipients. This mass of receivers, are often called as mass audience. The Mass audience is large, heterogenous and anonymous in nature. The receivers are scattered across a given village, state or country.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers文檔
• 文檔：交叉編碼器文檔
• 倉庫：GitHub上的Sentence Transformers
• Hugging Face：Hugging Face上的交叉編碼器

評估

指標

交叉編碼器重排序（數據集：gooaq-dev）

使用CrossEncoderRerankingEvaluator進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": false
}

交叉編碼器重排序（數據集：NanoMSMARCO_R100, NanoNFCorpus_R100, NanoNQ_R100）

使用CrossEncoderRerankingEvaluator進行評估，參數如下：

{
    "at_k": 10,
    "always_rerank_positives": true
}

交叉編碼器Nano BEIR（數據集：NanoBEIR_R100_mean）

使用CrossEncoderNanoBEIREvaluator進行評估，參數如下：

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

訓練詳情

訓練數據集

未命名數據集

• 大小：578,402個訓練樣本
• 列：question、answer和label
• 基於前1000個樣本的近似統計信息： | | 問題 | 答案 | 標籤 | |------|------|------|------| | 類型 | 字符串 | 字符串 | 整數 | | 詳情 |
  • 最小：17個字符
  • 平均：44.75個字符
  • 最大：84個字符
  |
  • 最小：54個字符
  • 平均：252.51個字符
  • 最大：388個字符
  |
  • 0：~83.00%
  • 1：~17.00%
  |
• 樣本： | 問題 | 答案 | 標籤 | |------|------|------| | how do you find mass? | Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass. | 1 | | how do you find mass? | In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers. | 0 | | how do you find mass? | A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV. | 0 |

使用的損失函數為BinaryCrossEntropyLoss，參數如下：

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 5
}

訓練超參數

非默認超參數

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

所有超參數

訓練日誌

加粗行表示保存的檢查點。

框架版本

• Python: 3.12.7
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

📄 許可證

本項目採用apache-2.0許可證。

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}