Nomic Embed Text V2 Moe Msmarco Bpr

🚀 基於nomic-ai/nomic-embed-text-v2-moe的句子轉換器

這是一個基於 sentence-transformers 從 nomic-ai/nomic-embed-text-v2-moe 微調而來的模型。它能將句子和段落映射到768維的密集向量空間，可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等任務。

🚀 快速開始

本模型可以方便地應用於多種自然語言處理任務，以下是使用步驟和示例代碼。

✨ 主要特性

• 高維映射：將句子和段落映射到768維的密集向量空間。
• 多任務適用：可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等。
• 長序列處理：最大序列長度可達8192個標記。

📦 安裝指南

首先，你需要安裝 Sentence Transformers 庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

from sentence_transformers import SentenceTransformer

# 從 🤗 Hub 下載模型
model = SentenceTransformer("BlackBeenie/nomic-embed-text-v2-moe-msmarco-bpr")
# 運行推理
sentences = [
    'what services are offered by adult day care',
    'Consumer Guide to Long Term Care. Adult Day Care. Adult day care is a planned program offered in a group setting that provides services that improve or maintain health or functioning, and social activities for seniors and persons with disabilities.',
    'The Met Life Market survey of 2008 on adult day services states the average cost for adult day care services is $64 per day. There has been an increase of 5% in these services in the past year.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# 獲取嵌入向量的相似度分數
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers 文檔
• 倉庫：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的 Sentence Transformers

完整模型架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

訓練詳情

訓練數據集

未命名數據集

• 規模：498,970 個訓練樣本
• 列名：sentence_0、sentence_1 和 sentence_2
• 近似統計（基於前1000個樣本）：

  	sentence_0	sentence_1	sentence_2
  類型	字符串	字符串	字符串
  詳情	最小：4個標記 平均：9.75個標記 最大：24個標記	最小：24個標記 平均：89.23個標記 最大：241個標記	最小：20個標記 平均：86.66個標記 最大：280個標記

• 樣本：

  sentence_0	sentence_1	sentence_2
  what the history of bluetooth	When asked about the name Bluetooth, I explained that Bluetooth was borrowed from the 10th century, second King of Denmark, King Harald Bluetooth; who was famous for uniting Scandinavia just as we intended to unite the PC and cellular industries with a short-range wireless link.	Technology: 1 How secure is a Bluetooth network? 2 What is Frequency-Hopping Spread Spectrum (FHSS)? 3 Will other RF (Radio Frequency) devices interfere with Bluetooth Devices? 4 Will Bluetooth and Wireless LAN (WLAN) interfere with each other? 5 What is the data throughput speed of a Bluetooth connection? 6 What is the range of Bluetooth 7 ... What kind of ...
  how thin can a concrete slab be	Another issue that must be addressed is the added weight of the thin-slab. Poured gypsum thin-slabs typically add 13 to 15 pounds per square foot to the dead loading of a floor structure. Standard weight concrete thin slabs add about 18 pounds per square foot (at 1.5 thickness).	Find the Area in square feet: We will use a concrete slab pour for our example. Let’s say that we need to figure out the yardage for a slab that will be 15 feet long by 10 feet wide and 4 inches thick. First we find the area by multiplying the length times the width. 1 15 feet X 10 feet = 150 square feet.
  how long to cook eggs to hard boil	This method works best if the eggs are in a single layer, but you can double them up as well, you'll just need to add more time to the steaming time. 3 Set your timer for 6 minutes for soft boiled, 10 minutes for hard boiled with a still translucent and bright yolk, or 12-15 minutes for cooked-through hard boiled.	Hard-Steamed Eggs. Fill a pot that can comfortably hold your steamer with the lid on with 1 to 2 inches of water. Bring to a rolling boil, 212 degrees Fahrenheit. Place your eggs in a metal steamer, and lower the basket into the pot. The eggs should sit above the boiling water. Cover and cook for 12 minutes. Hard-steamed eggs, like hard-boiled eggs, are eggs that are cooked until the egg yolk is fully set and has turned to a chalky texture.

• 損失函數：beir.losses.bpr_loss.BPRLoss

訓練超參數

非默認超參數

• eval_strategy: steps
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 5
• fp16: True
• multi_dataset_batch_sampler: round_robin

所有超參數

訓練日誌

框架版本

• Python: 3.11.11
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

📄 許可證

文檔中未提及相關許可證信息。

📄 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}