Modernbert Large Msmarco Bpr

🚀 SentenceTransformer based on answerdotai/ModernBERT-large

このモデルは、answerdotai/ModernBERT-large から微調整された sentence-transformers モデルです。文章や段落を1024次元の密ベクトル空間にマッピングし、意味的な文章類似度の計算、意味検索、言い換えマイニング、文章分類、クラスタリングなどに使用できます。

📦 インストール

まず、Sentence Transformersライブラリをインストールします。

pip install -U sentence-transformers

💻 使用例

基本的な使用法

from sentence_transformers import SentenceTransformer

# 🤗 Hubからダウンロード
model = SentenceTransformer("BlackBeenie/ModernBERT-large-msmarco-bpr")
# 推論を実行
sentences = [
    'what is the average top third score on the act',
    'North Dakota is among a dozen states where high school students are required to take the ACT before graduating. The state tied with Colorado for third with an average composite score of 20.6 this year. Utah was first with an average of 20.8 and Illinois was second at 20.7. ACT composite scores range from 1 to 36. The national average is 21.0. A total of 7,227 students in North Dakota took the ACT this year.',
    "The average ACT score composite at Duke is a 34. The 25th percentile ACT score is 32, and the 75th percentile ACT score is 35. In other words, a 32 places you below average, while a 35 will move you up to above average.f you're a junior or senior, your GPA is hard to change from this point on. If your GPA is at or below the school average of 4.19, you'll need a higher ACT score to compensate and show that you're prepared to take on college academics.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# 埋め込みベクトルの類似度スコアを取得
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 ドキュメント

モデルの詳細

モデルの説明

モデルのソース

• ドキュメント: Sentence Transformers Documentation
• リポジトリ: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

モデルのアーキテクチャ

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

学習の詳細

学習データセット

無名データセット

• サイズ: 498,970個の学習サンプル
• 列: sentence_0、sentence_1、sentence_2
• 最初の1000サンプルに基づく概算統計情報:

  	sentence_0	sentence_1	sentence_2
  タイプ	string	string	string
  詳細	最小: 4トークン 平均: 9.24トークン 最大: 27トークン	最小: 23トークン 平均: 83.71トークン 最大: 279トークン	最小: 17トークン 平均: 79.72トークン 最大: 262トークン

• サンプル:

  sentence_0	sentence_1	sentence_2
  what is tongkat ali	Tongkat Ali is a very powerful herb that acts as a sex enhancer by naturally increasing the testosterone levels, and revitalizing sexual impotence, performance and pleasure. Tongkat Ali is also effective in building muscular volume & strength resulting to a healthy physique.	However, unlike tongkat ali extract, tongkat ali chipped root and root powder are not sterile. Thus, the raw consumption of root powder is not recommended. The traditional preparation in Indonesia and Malaysia is to boil chipped roots as a tea. A standard dosage would be 50 gram of chipped root per person per day.
  cost to install engineered hardwood flooring	Burton says his customers typically spend about $8 per square foot for engineered hardwood flooring; add an additional $2 per square foot for installation. Minion says consumers should expect to pay $7 to $12 per square foot for quality hardwood flooring. “If the homeowner buys the wood and you need somebody to install it, usually an installation goes for about $2 a square foot,” Bill LeBeau, owner of LeBeau’s Hardwood Floors of Huntersville, North Carolina, says.	Installing hardwood flooring can cost between $9 and $12 per square foot, compared with about $3 to $5 per square foot for carpet—so some homeowners opt to install hardwood only in some rooms rather than throughout their home.However, carpet typically needs to be replaced if it becomes stained or worn out.ardwood flooring lasts longer than carpet, can be easier to keep clean and can be refinished. In the end, though, the decision about whether to install hardwood or carpeting in a bedroom should be based on your personal preference, at least if you intend to stay in the home for years.
  define pollute	pollutes; polluted; polluting. Learner's definition of POLLUTE. [+ object] : to make (land, water, air, etc.) dirty and not safe or suitable to use. Waste from the factory had polluted [=contaminated] the river. Miles of beaches were polluted by the oil spill. Car exhaust pollutes the air.	Definition of pollute written for English Language Learners from the Merriam-Webster Learner's Dictionary with audio pronunciations, usage examples, and count/noncount noun labels. Learner's Dictionary mobile search

• 損失関数: beir.losses.bpr_loss.BPRLoss

学習ハイパーパラメータ

デフォルトではないハイパーパラメータ

• eval_strategy: steps
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 5
• fp16: True
• multi_dataset_batch_sampler: round_robin

すべてのハイパーパラメータ