Reranker ModernBERT Base Gooaq Bce

🚀 GooAQで学習されたModernBERT-base

このモデルは、sentence-transformersライブラリを使用して、answerdotai/ModernBERT-baseからファインチューニングされたCross Encoderモデルです。テキストのペアに対するスコアを計算し、テキストの再ランキングや意味検索に利用できます。

🚀 クイックスタート

このモデルは、テキストのペアに対するスコアを計算し、テキストの再ランキングや意味検索に利用できます。以下のセクションでは、モデルの詳細、使用方法、評価結果、学習の詳細について説明します。

✨ 主な機能

• テキストのペアに対するスコアを計算します。
• テキストの再ランキングや意味検索に利用できます。
• 最大8192トークンのシーケンスを処理できます。

📦 インストール

まず、Sentence Transformersライブラリをインストールします。

pip install -U sentence-transformers

💻 使用例

基本的な使用法

from sentence_transformers import CrossEncoder

# 🤗 Hubからダウンロード
model = CrossEncoder("akr2002/reranker-ModernBERT-base-gooaq-bce")
# テキストのペアに対するスコアを取得
pairs = [
    ['how do you find mass?', "Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass."],
    ['how do you find mass?', "In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers."],
    ['how do you find mass?', 'A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV.'],
    ['how do you find mass?', 'Mass is the amount of matter in a body, normally measured in grams or kilograms etc. Weight is a force that pulls on a mass and is measured in Newtons. ... Density basically means how much mass is occupied in a specific volume or space. Different materials of the same size may have different masses because of its density.'],
    ['how do you find mass?', 'Receiver – Mass communication is the transmission of the message to a large number of recipients. This mass of receivers, are often called as mass audience. The Mass audience is large, heterogenous and anonymous in nature. The receivers are scattered across a given village, state or country.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# 単一のテキストとの類似度に基づいて、異なるテキストをランク付けする
ranks = model.rank(
    'how do you find mass?',
    [
        "Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass.",
        "In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers.",
        'A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV.',
        'Mass is the amount of matter in a body, normally measured in grams or kilograms etc. Weight is a force that pulls on a mass and is measured in Newtons. ... Density basically means how much mass is occupied in a specific volume or space. Different materials of the same size may have different masses because of its density.',
        'Receiver – Mass communication is the transmission of the message to a large number of recipients. This mass of receivers, are often called as mass audience. The Mass audience is large, heterogenous and anonymous in nature. The receivers are scattered across a given village, state or country.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 ドキュメント

モデルの詳細

モデルの説明

モデルのソース

• ドキュメント: Sentence Transformers Documentation
• ドキュメント: Cross Encoder Documentation
• リポジトリ: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

評価

メトリクス

Cross Encoder Reranking

• データセット: gooaq-dev
• 評価は、以下のパラメータでCrossEncoderRerankingEvaluatorを使用して行われました。

  {
      "at_k": 10,
      "always_rerank_positives": false
  }
  

Cross Encoder Reranking

• データセット: NanoMSMARCO_R100, NanoNFCorpus_R100 および NanoNQ_R100
• 評価は、以下のパラメータでCrossEncoderRerankingEvaluatorを使用して行われました。

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Cross Encoder Nano BEIR

• データセット: NanoBEIR_R100_mean
• 評価は、以下のパラメータでCrossEncoderNanoBEIREvaluatorを使用して行われました。

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

学習の詳細

学習データセット

無名データセット

• サイズ: 578,402個の学習サンプル
• 列: question, answer, および label
• 最初の1000サンプルに基づく概算統計:

  	質問	回答	ラベル
  タイプ	文字列	文字列	整数
  詳細	最小: 17文字 平均: 44.75文字 最大: 84文字	最小: 54文字 平均: 252.51文字 最大: 388文字	0: ~83.00% 1: ~17.00%

• サンプル:

  質問	回答	ラベル
  how do you find mass?	Divide the object's weight by the acceleration of gravity to find the mass. You'll need to convert the weight units to Newtons. For example, 1 kg = 9.807 N. If you're measuring the mass of an object on Earth, divide the weight in Newtons by the acceleration of gravity on Earth (9.8 meters/second2) to get mass.	1
  how do you find mass?	In general use, 'High Mass' means a full ceremonial Mass, most likely with music, and also with incense if they're particularly traditional. ... Incense is used quite a lot. Low Mass in the traditional rite is celebrated by one priest, and usually only one or two altar servers.	0
  how do you find mass?	A neutron has a slightly larger mass than the proton. These are often given in terms of an atomic mass unit, where one atomic mass unit (u) is defined as 1/12th the mass of a carbon-12 atom. You can use that to prove that a mass of 1 u is equivalent to an energy of 931.5 MeV.	0

• 損失関数: BinaryCrossEntropyLoss を以下のパラメータで使用。

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": 5
  }
  

学習ハイパーパラメータ

非デフォルトのハイパーパラメータ

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

すべてのハイパーパラメータ

学習ログ

• 太字の行は、保存されたチェックポイントを示しています。

フレームワークのバージョン

• Python: 3.12.7
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

📄 ライセンス

このモデルは、Apache 2.0ライセンスの下で提供されています。

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}