Finance2 Embedding Small En V1.5

🚀 基於BAAI/bge-small-en-v1.5的句子轉換器

本模型是基於 Sentence Transformers 框架，在 baconnier/finance_dataset_small_private 數據集上對 BAAI/bge-small-en-v1.5 進行微調得到的。它能夠將句子和段落映射到384維的密集向量空間，可用於語義文本相似度計算、語義搜索、釋義挖掘、文本分類、聚類等任務。

✨ 主要特性

• 基於預訓練模型 BAAI/bge-small-en-v1.5 進行微調，在金融領域數據集上訓練，能更好地處理金融相關文本。
• 可將文本映射到384維的密集向量空間，便於進行語義相似度計算等任務。
• 支持多種相似度計算函數，如餘弦相似度。

📦 安裝指南

首先，安裝 Sentence Transformers 庫：

pip install -U sentence-transformers

💻 使用示例

基礎用法

from sentence_transformers import SentenceTransformer

# 從 🤗 Hub 下載模型
model = SentenceTransformer("baconnier/Finance2_embedding_small_en-V1.5")
# 進行推理
sentences = [
    'What is industrial production, and how is it measured by the Federal Reserve Board?',
    'Industrial production is a statistic determined by the Federal Reserve Board that measures the total output of all US factories and mines on a monthly basis. The Fed collects data from various government agencies and trade associations to calculate the industrial production index, which serves as an important economic indicator, providing insight into the health of the manufacturing and mining sectors.\nIndustrial production is a monthly statistic calculated by the Federal Reserve Board, measuring the total output of US factories and mines using data from government agencies and trade associations, serving as a key economic indicator for the manufacturing and mining sectors.',
    'Industrial production is a statistic that measures the output of factories and mines in the US. It is released by the Federal Reserve Board every quarter.\nIndustrial production measures factory and mine output, released quarterly by the Fed.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# 獲取嵌入向量的相似度分數
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers 文檔
• 倉庫：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的 Sentence Transformers

完整模型架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

評估

指標

三元組

• 數據集：Finance_Embedding_Metric
• 使用 TripletEvaluator 進行評估

訓練詳情

訓練數據集

• 數據集：baconnier/finance_dataset_small_private，版本 d7e6492
• 大小：15,525 個訓練樣本
• 列：anchor、positive 和 negative
• 損失函數：MultipleNegativesRankingLoss，參數如下：

{
    "scale": 20.0,
    "similarity_fct": "cos_sim"
}

評估數據集

• 數據集：baconnier/finance_dataset_small_private，版本 d7e6492
• 大小：862 個評估樣本
• 列：anchor、positive 和 negative
• 損失函數：MultipleNegativesRankingLoss，參數如下：

{
    "scale": 20.0,
    "similarity_fct": "cos_sim"
}

訓練超參數

非默認超參數

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 1
• warmup_ratio: 0.1
• bf16: True
• batch_sampler: no_duplicates

訓練日誌

框架版本

• Python: 3.10.12
• Sentence Transformers: 3.0.1
• Transformers: 4.41.2
• PyTorch: 2.3.0+cu121
• Accelerate: 0.31.0
• Datasets: 2.19.2
• Tokenizers: 0.19.1

📄 許可證

文檔中未提及相關許可證信息。

📖 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}