Mind Map Blog Model

🚀 基於sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2的句子轉換器模型

本項目基於 sentence-transformers 框架，對 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 模型進行微調。該模型可將句子和段落映射到384維的密集向量空間，適用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等多種自然語言處理任務。

🚀 快速開始

安裝依賴庫

首先，你需要安裝 sentence-transformers 庫：

pip install -U sentence-transformers

加載模型並進行推理

from sentence_transformers import SentenceTransformer

# 從 🤗 Hub 下載模型
model = SentenceTransformer("sentence_transformers_model_id")
# 進行推理
sentences = [
    "What was Nikola Tesla's IQ?",
    'Did Nikola Tesla have children?',
    'What hotel in Raipur would be safe for unmarried couples, without the harassment of police, hotel staff, and moral police?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# 獲取嵌入向量的相似度分數
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

✨ 主要特性

• 多語言支持：基於多語言的基礎模型，能夠處理多種語言的句子和段落。
• 高效的向量映射：將文本映射到384維的密集向量空間，便於進行語義相似度計算和其他自然語言處理任務。
• 多種應用場景：可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等。

📦 安裝指南

安裝 sentence-transformers 庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

from sentence_transformers import SentenceTransformer

# 從 🤗 Hub 下載模型
model = SentenceTransformer("sentence_transformers_model_id")
# 進行推理
sentences = [
    "What was Nikola Tesla's IQ?",
    'Did Nikola Tesla have children?',
    'What hotel in Raipur would be safe for unmarried couples, without the harassment of police, hotel staff, and moral police?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# 獲取嵌入向量的相似度分數
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers 文檔
• 代碼倉庫：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的 Sentence Transformers

完整模型架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

訓練詳情

訓練數據集

• 未命名數據集
  • 規模：101,072 個訓練樣本
  • 列名：sentence_0、sentence_1 和 label
  • 基於前1000個樣本的近似統計信息：

    	sentence_0	sentence_1	label
    類型	字符串	字符串	浮點數
    詳情	最小值: 6 個詞元 平均值: 16.52 個詞元 最大值: 59 個詞元	最小值: 6 個詞元 平均值: 16.55 個詞元 最大值: 78 個詞元	最小值: 0.0 平均值: 0.37 最大值: 1.0

  • 樣本示例：

    sentence_0	sentence_1	label
    How can I improve my pronunciation of English words?	How can I improve my pronunciation in English?	1.0
    How can I increase my attractiveness?	How do I increase my attraction towards someone?	0.0
    What are the best places to work in?	Which are the best places to work in india?	0.0

  • 損失函數：MultipleNegativesRankingLoss，參數如下：

    {
        "scale": 20.0,
        "similarity_fct": "cos_sim"
    }
    

訓練超參數

非默認超參數

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 1
• multi_dataset_batch_sampler: round_robin

所有超參數

訓練日誌

框架版本

• Python: 3.9.20
• Sentence Transformers: 3.1.1
• Transformers: 4.45.1
• PyTorch: 2.4.1+cpu
• Accelerate: 0.34.2
• Datasets: 3.0.1
• Tokenizers: 0.20.0

📄 許可證

本項目的相關引用信息如下：

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}