Modernbert Large Msmarco Bpr

🚀 基於answerdotai/ModernBERT-large的句子轉換器

這是一個基於 answerdotai/ModernBERT-large 微調的 sentence-transformers 模型。它可以將句子和段落映射到一個1024維的密集向量空間，可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等任務。

🚀 快速開始

本模型可以將句子和段落映射到一個1024維的密集向量空間，可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等任務。

✨ 主要特性

• 基於 answerdotai/ModernBERT-large 進行微調。
• 能夠將句子和段落映射到1024維的密集向量空間。
• 可用於多種自然語言處理任務，如語義文本相似度計算、語義搜索等。

📦 安裝指南

首先，你需要安裝 Sentence Transformers 庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

from sentence_transformers import SentenceTransformer

# 從 🤗 Hub 下載模型
model = SentenceTransformer("BlackBeenie/ModernBERT-large-msmarco-bpr")
# 運行推理
sentences = [
    'what is the average top third score on the act',
    'North Dakota is among a dozen states where high school students are required to take the ACT before graduating. The state tied with Colorado for third with an average composite score of 20.6 this year. Utah was first with an average of 20.8 and Illinois was second at 20.7. ACT composite scores range from 1 to 36. The national average is 21.0. A total of 7,227 students in North Dakota took the ACT this year.',
    "The average ACT score composite at Duke is a 34. The 25th percentile ACT score is 32, and the 75th percentile ACT score is 35. In other words, a 32 places you below average, while a 35 will move you up to above average.f you're a junior or senior, your GPA is hard to change from this point on. If your GPA is at or below the school average of 4.19, you'll need a higher ACT score to compensate and show that you're prepared to take on college academics.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# 獲取嵌入向量的相似度分數
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers 文檔
• 倉庫：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的 Sentence Transformers

完整模型架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

訓練詳情

訓練數據集

未命名數據集

• 大小：498,970 個訓練樣本
• 列：sentence_0、sentence_1 和 sentence_2
• 基於前1000個樣本的近似統計信息：

  	sentence_0	sentence_1	sentence_2
  類型	字符串	字符串	字符串
  詳情	最小：4 個標記 平均：9.24 個標記 最大：27 個標記	最小：23 個標記 平均：83.71 個標記 最大：279 個標記	最小：17 個標記 平均：79.72 個標記 最大：262 個標記

• 樣本：

  sentence_0	sentence_1	sentence_2
  what is tongkat ali	Tongkat Ali is a very powerful herb that acts as a sex enhancer by naturally increasing the testosterone levels, and revitalizing sexual impotence, performance and pleasure. Tongkat Ali is also effective in building muscular volume & strength resulting to a healthy physique.	However, unlike tongkat ali extract, tongkat ali chipped root and root powder are not sterile. Thus, the raw consumption of root powder is not recommended. The traditional preparation in Indonesia and Malaysia is to boil chipped roots as a tea. A standard dosage would be 50 gram of chipped root per person per day.
  cost to install engineered hardwood flooring	Burton says his customers typically spend about $8 per square foot for engineered hardwood flooring; add an additional $2 per square foot for installation. Minion says consumers should expect to pay $7 to $12 per square foot for quality hardwood flooring. “If the homeowner buys the wood and you need somebody to install it, usually an installation goes for about $2 a square foot,” Bill LeBeau, owner of LeBeau’s Hardwood Floors of Huntersville, North Carolina, says.	Installing hardwood flooring can cost between $9 and $12 per square foot, compared with about $3 to $5 per square foot for carpet—so some homeowners opt to install hardwood only in some rooms rather than throughout their home.However, carpet typically needs to be replaced if it becomes stained or worn out.ardwood flooring lasts longer than carpet, can be easier to keep clean and can be refinished. In the end, though, the decision about whether to install hardwood or carpeting in a bedroom should be based on your personal preference, at least if you intend to stay in the home for years.
  define pollute	pollutes; polluted; polluting. Learner's definition of POLLUTE. [+ object] : to make (land, water, air, etc.) dirty and not safe or suitable to use. Waste from the factory had polluted [=contaminated] the river. Miles of beaches were polluted by the oil spill. Car exhaust pollutes the air.	Definition of pollute written for English Language Learners from the Merriam-Webster Learner's Dictionary with audio pronunciations, usage examples, and count/noncount noun labels. Learner's Dictionary mobile search

• 損失：beir.losses.bpr_loss.BPRLoss

訓練超參數

非默認超參數

• eval_strategy: steps
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 5
• fp16: True
• multi_dataset_batch_sampler: round_robin

所有超參數

訓練日誌

框架版本

• Python: 3.11.11
• Sentence Transformers: 3.4.1
• Transformers: 4.48.2
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.2.0
• Tokenizers: 0.21.0

📄 許可證

文檔中未提及相關信息。

📖 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}