Bge M3 Financial Matryoshka
基于BAAI/bge-m3微调的金融领域句子嵌入模型,支持1024维向量表示,适用于语义相似度和信息检索任务
下载量 34
发布时间 : 6/22/2024
模型简介
这是一个基于sentence-transformers框架的金融领域专用模型,能够将文本映射到高维向量空间,主要用于语义文本相似度计算、信息检索等任务。模型在金融文本上进行了优化,特别适合处理财务报告、商业文档等专业内容。
模型特点
金融领域优化
针对金融文本进行专门微调,在处理财务报告、商业文档等专业内容时表现更优
高维向量表示
支持1024维的密集向量表示,能够捕捉更丰富的语义信息
多维度评估
模型在多个维度(1024/768/512/384)上都进行了评估,确保不同场景下的性能
长文本支持
最大序列长度达8192个标记,适合处理较长的金融文档
模型能力
语义文本相似度计算
语义搜索
复述挖掘
文本分类
文本聚类
金融文档分析
使用案例
金融文档处理
财务报告检索
从大量财务报告中快速找到与查询问题最相关的段落
在测试集上达到71.7%的准确率@1
监管文档分析
分析监管文件中特定条款的相关内容
在测试集上达到83.1%的准确率@3
商业智能
商业报告相似度分析
比较不同商业报告中的相似内容
🚀 BGE-M3 Financial Matryoshka
BGE-M3 Financial Matryoshka 是一个基于 sentence-transformers 框架,从 BAAI/bge-m3 微调而来的模型。它能够将句子和段落映射到一个 1024 维的密集向量空间,可用于语义文本相似度计算、语义搜索、释义挖掘、文本分类、聚类等多种自然语言处理任务。
🚀 快速开始
安装依赖库
首先,你需要安装 sentence-transformers
库:
pip install -U sentence-transformers
加载模型并进行推理
安装完成后,你可以加载该模型并进行推理:
from sentence_transformers import SentenceTransformer
# 从 🤗 Hub 下载模型
model = SentenceTransformer("haophancs/bge-m3-financial-matryoshka")
# 进行推理
sentences = [
'As of January 28, 2024 the net carrying value of our inventories was $1.3 billion, which included provisions for obsolete and damaged inventory of $139.7 million.',
"What is the status of the company's inventory as of January 28, 2024, in terms of its valuation and provisions for obsolescence?",
'What is the relationship between the ESG goals and the long - term growth strategy?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# 获取嵌入向量的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
✨ 主要特性
- 多任务适用性:可用于语义文本相似度计算、语义搜索、释义挖掘、文本分类、聚类等多种自然语言处理任务。
- 高维向量表示:能够将句子和段落映射到 1024 维的密集向量空间,更好地捕捉语义信息。
- 长序列处理:最大序列长度可达 8192 个标记,适合处理较长的文本。
📦 安装指南
安装 sentence-transformers
库即可使用该模型:
pip install -U sentence-transformers
💻 使用示例
基础用法
from sentence_transformers import SentenceTransformer
# 从 🤗 Hub 下载模型
model = SentenceTransformer("haophancs/bge-m3-financial-matryoshka")
# 进行推理
sentences = [
'As of January 28, 2024 the net carrying value of our inventories was $1.3 billion, which included provisions for obsolete and damaged inventory of $139.7 million.',
"What is the status of the company's inventory as of January 28, 2024, in terms of its valuation and provisions for obsolescence?",
'What is the relationship between the ESG goals and the long - term growth strategy?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# 获取嵌入向量的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
📚 详细文档
模型详情
模型描述
属性 | 详情 |
---|---|
模型类型 | 句子转换器(Sentence Transformer) |
基础模型 | BAAI/bge-m3 |
最大序列长度 | 8192 个标记 |
输出维度 | 1024 个标记 |
相似度函数 | 余弦相似度(Cosine Similarity) |
语言 | 英语(en) |
许可证 | apache-2.0 |
模型来源
- 文档:Sentence Transformers Documentation
- 代码仓库:Sentence Transformers on GitHub
- Hugging Face:Sentence Transformers on Hugging Face
完整模型架构
SentenceTransformer(
(0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
评估
信息检索指标
以下是在不同数据集上的评估指标:
dim_1024
数据集
指标 | 值 |
---|---|
cosine_accuracy@1 | 0.7171 |
cosine_accuracy@3 | 0.8314 |
cosine_accuracy@5 | 0.87 |
cosine_accuracy@10 | 0.9143 |
cosine_precision@1 | 0.7171 |
cosine_precision@3 | 0.2771 |
cosine_precision@5 | 0.174 |
cosine_precision@10 | 0.0914 |
cosine_recall@1 | 0.7171 |
cosine_recall@3 | 0.8314 |
cosine_recall@5 | 0.87 |
cosine_recall@10 | 0.9143 |
cosine_ndcg@10 | 0.8152 |
cosine_mrr@10 | 0.7836 |
cosine_map@100 | 0.7867 |
dim_768
数据集
指标 | 值 |
---|---|
cosine_accuracy@1 | 0.7129 |
cosine_accuracy@3 | 0.8343 |
cosine_accuracy@5 | 0.8657 |
cosine_accuracy@10 | 0.91 |
cosine_precision@1 | 0.7129 |
cosine_precision@3 | 0.2781 |
cosine_precision@5 | 0.1731 |
cosine_precision@10 | 0.091 |
cosine_recall@1 | 0.7129 |
cosine_recall@3 | 0.8343 |
cosine_recall@5 | 0.8657 |
cosine_recall@10 | 0.91 |
cosine_ndcg@10 | 0.8122 |
cosine_mrr@10 | 0.7809 |
cosine_map@100 | 0.7843 |
dim_512
数据集
指标 | 值 |
---|---|
cosine_accuracy@1 | 0.7114 |
cosine_accuracy@3 | 0.8357 |
cosine_accuracy@5 | 0.8643 |
cosine_accuracy@10 | 0.91 |
cosine_precision@1 | 0.7114 |
cosine_precision@3 | 0.2786 |
cosine_precision@5 | 0.1729 |
cosine_precision@10 | 0.091 |
cosine_recall@1 | 0.7114 |
cosine_recall@3 | 0.8357 |
cosine_recall@5 | 0.8643 |
cosine_recall@10 | 0.91 |
cosine_ndcg@10 | 0.811 |
cosine_mrr@10 | 0.7793 |
cosine_map@100 | 0.7827 |
dim_384
数据集
指标 | 值 |
---|---|
cosine_accuracy@1 | 0.7143 |
cosine_accuracy@3 | 0.8329 |
cosine_accuracy@5 | 0.8629 |
cosine_accuracy@10 | 0.9129 |
cosine_precision@1 | 0.7143 |
cosine_precision@3 | 0.2776 |
cosine_precision@5 | 0.1726 |
cosine_precision@10 | 0.0913 |
cosine_recall@1 | 0.7143 |
cosine_recall@3 | 0.8329 |
cosine_recall@5 | 0.8629 |
cosine_recall@10 | 0.9129 |
cosine_ndcg@10 | 0.8126 |
cosine_mrr@10 | 0.7806 |
cosine_map@100 | 0.7838 |
训练详情
训练数据集
- 未命名数据集
- 数据规模:6300 个训练样本
- 数据列:
<code>positive</code>
和<code>anchor</code>
- 近似统计信息(基于前 1000 个样本):
positive anchor 类型 字符串 字符串 详情 - 最小:11 个标记
- 平均:51.97 个标记
- 最大:1146 个标记
- 最小:7 个标记
- 平均:21.63 个标记
- 最大:47 个标记
- 样本示例:
positive anchor From fiscal year 2022 to 2023, the cost of revenue as a percentage of total net revenue decreased by 3 percent.
What was the percentage change in cost of revenue as a percentage of total net revenue from fiscal year 2022 to 2023?
•Operating income increased $321 million, or 2%, to $18.1 billion versus year ago due to the increase in net sales, partially offset by a modest decrease in operating margin.
What factors contributed to the increase in operating income for Procter & Gamble in 2023?
market specific brands including 'Aurrera,' 'Lider,' and 'PhonePe.'
What specific brands does Walmart International market?
- 损失函数:
MatryoshkaLoss
,参数如下:{ "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 1024, 768, 512, 384 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 }
训练超参数
- 非默认超参数:
eval_strategy
: epochper_device_train_batch_size
: 4per_device_eval_batch_size
: 2gradient_accumulation_steps
: 2learning_rate
: 2e-05num_train_epochs
: 4lr_scheduler_type
: cosinewarmup_ratio
: 0.1bf16
: Truetf32
: Trueload_best_model_at_end
: Trueoptim
: adamw_torch_fusedbatch_sampler
: no_duplicates
框架版本
- Python: 3.12.2
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.2.0+cu121
- Accelerate: 0.31.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1
🔧 技术细节
训练日志
点击展开
轮次 | 步数 | 训练损失 | dim_1024_cosine_map@100 | dim_384_cosine_map@100 | dim_512_cosine_map@100 | dim_768_cosine_map@100 |
---|---|---|---|---|---|---|
0.0127 | 10 | 0.2059 | - | - | - | - |
0.0254 | 20 | 0.2612 | - | - | - | - |
0.0381 | 30 | 0.0873 | - | - | - | - |
0.0508 | 40 | 0.1352 | - | - | - | - |
0.0635 | 50 | 0.156 | - | - | - | - |
0.0762 | 60 | 0.0407 | - | - | - | - |
0.0889 | 70 | 0.09 | - | - | - | - |
0.1016 | 80 | 0.027 | - | - | - | - |
0.1143 | 90 | 0.0978 | - | - | - | - |
0.1270 | 100 | 0.0105 | - | - | - | - |
0.1397 | 110 | 0.0402 | - | - | - | - |
0.1524 | 120 | 0.0745 | - | - | - | - |
0.1651 | 130 | 0.0655 | - | - | - | - |
0.1778 | 140 | 0.0075 | - | - | - | - |
0.1905 | 150 | 0.0141 | - | - | - | - |
0.2032 | 160 | 0.0615 | - | - | - | - |
0.2159 | 170 | 0.0029 | - | - | - | - |
0.2286 | 180 | 0.0269 | - | - | - | - |
0.2413 | 190 | 0.0724 | - | - | - | - |
0.2540 | 200 | 0.0218 | - | - | - | - |
0.2667 | 210 | 0.0027 | - | - | - | - |
0.2794 | 220 | 0.007 | - | - | - | - |
0.2921 | 230 | 0.0814 | - | - | - | - |
0.3048 | 240 | 0.0326 | - | - | - | - |
0.3175 | 250 | 0.0061 | - | - | - | - |
0.3302 | 260 | 0.0471 | - | - | - | - |
0.3429 | 270 | 0.0115 | - | - | - | - |
0.3556 | 280 | 0.0021 | - | - | - | - |
0.3683 | 290 | 0.0975 | - | - | - | - |
0.3810 | 300 | 0.0572 | - | - | - | - |
0.3937 | 310 | 0.0125 | - | - | - | - |
0.4063 | 320 | 0.04 | - | - | - | - |
0.4190 | 330 | 0.0023 | - | - | - | - |
0.4317 | 340 | 0.0121 | - | - | - | - |
0.4444 | 350 | 0.0116 | - | - | - | - |
0.4571 | 360 | 0.0059 | - | - | - | - |
0.4698 | 370 | 0.0217 | - | - | - | - |
0.4825 | 380 | 0.0294 | - | - | - | - |
0.4952 | 390 | 0.1102 | - | - | - | - |
0.5079 | 400 | 0.0103 | - | - | - | - |
0.5206 | 410 | 0.0023 | - | - | - | - |
0.5333 | 420 | 0.0157 | - | - | - | - |
0.5460 | 430 | 0.0805 | - | - | - | - |
0.5587 | 440 | 0.0168 | - | - | - | - |
0.5714 | 450 | 0.1279 | - | - | - | - |
0.5841 | 460 | 0.2012 | - | - | - | - |
0.5968 | 470 | 0.0436 | - | - | - | - |
0.6095 | 480 | 0.0204 | - | - | - | - |
0.6222 | 490 | 0.0097 | - | - | - | - |
0.6349 | 500 | 0.0013 | - | - | - | - |
0.6476 | 510 | 0.0042 | - | - | - | - |
0.6603 | 520 | 0.0034 | - | - | - | - |
0.6730 | 530 | 0.0226 | - | - | - | - |
0.6857 | 540 | 0.0267 | - | - | - | - |
0.6984 | 550 | 0.0007 | - | - | - | - |
0.7111 | 560 | 0.0766 | - | - | - | - |
0.7238 | 570 | 0.2174 | - | - | - | - |
0.7365 | 580 | 0.0089 | - | - | - | - |
0.7492 | 590 | 0.0794 | - | - | - | - |
0.7619 | 600 | 0.0031 | - | - | - | - |
0.7746 | 610 | 0.0499 | - | - | - | - |
0.7873 | 620 | 0.0105 | - | - | - | - |
0.8 | 630 | 0.0097 | - | - | - | - |
0.8127 | 640 | 0.0028 | - | - | - | - |
0.8254 | 650 | 0.0029 | - | - | - | - |
0.8381 | 660 | 0.1811 | - | - | - | - |
0.8508 | 670 | 0.064 | - | - | - | - |
0.8635 | 680 | 0.0139 | - | - | - | - |
0.8762 | 690 | 0.055 | - | - | - | - |
0.8889 | 700 | 0.0013 | - | - | - | - |
0.9016 | 710 | 0.0402 | - | - | - | - |
0.9143 | 720 | 0.0824 | - | - | - | - |
0.9270 | 730 | 0.03 | - | - | - | - |
0.9397 | 740 | 0.0337 | - | - | - | - |
0.9524 | 750 | 0.1192 | - | - | - | - |
0.9651 | 760 | 0.0039 | - | - | - | - |
0.9778 | 770 | 0.004 | - | - | - | - |
0.9905 | 780 | 0.1413 | - | - | - | - |
0.9994 | 787 | - | 0.7851 | 0.7794 | 0.7822 | 0.7863 |
1.0032 | 790 | 0.019 | - | - | - | - |
1.0159 | 800 | 0.0587 | - | - | - | - |
1.0286 | 810 | 0.0186 | - | - | - | - |
1.0413 | 820 | 0.0018 | - | - | - | - |
1.0540 | 830 | 0.0631 | - | - | - | - |
1.0667 | 840 | 0.0127 | - | - | - | - |
1.0794 | 850 | 0.0037 | - | - | - | - |
1.0921 | 860 | 0.0029 | - | - | - | - |
1.1048 | 870 | 0.1437 | - | - | - | - |
1.1175 | 880 | 0.0015 | - | - | - | - |
1.1302 | 890 | 0.0024 | - | - | - | - |
1.1429 | 900 | 0.0133 | - | - | - | - |
1.1556 | 910 | 0.0245 | - | - | - | - |
1.1683 | 920 | 0.0017 | - | - | - | - |
1.1810 | 930 | 0.0007 | - | - | - | - |
1.1937 | 940 | 0.002 | - | - | - | - |
1.2063 | 950 | 0.0044 | - | - | - | - |
1.2190 | 960 | 0.0009 | - | - | - | - |
1.2317 | 970 | 0.01 | - | - | - | - |
1.2444 | 980 | 0.0026 | - | - | - | - |
1.2571 | 990 | 0.0017 | - | - | - | - |
1.2698 | 1000 | 0.0014 | - | - | - | - |
1.2825 | 1010 | 0.0009 | - | - | - | - |
1.2952 | 1020 | 0.0829 | - | - | - | - |
1.3079 | 1030 | 0.0011 | - | - | - | - |
1.3206 | 1040 | 0.012 | - | - | - | - |
1.3333 | 1050 | 0.0019 | - | - | - | - |
1.3460 | 1060 | 0.0007 | - | - | - | - |
1.3587 | 1070 | 0.0141 | - | - | - | - |
1.3714 | 1080 | 0.0003 | - | - | - | - |
1.3841 | 1090 | 0.001 | - | - | - | - |
1.3968 | 1100 | 0.0005 | - | - | - | - |
1.4095 | 1110 | 0.0031 | - | - | - | - |
1.4222 | 1120 | 0.0004 | - | - | - | - |
1.4349 | 1130 | 0.0054 | - | - | - | - |
1.4476 | 1140 | 0.0003 | - | - | - | - |
1.4603 | 1150 | 0.0007 | - | - | - | - |
1.4730 | 1160 | 0.0009 | - | - | - | - |
1.4857 | 1170 | 0.001 | - | - | - | - |
1.4984 | 1180 | 0.0006 | - | - | - | - |
1.5111 | 1190 | 0.0046 | - | - | - | - |
1.5238 | 1200 | 0.0003 | - | - | - | - |
1.5365 | 1210 | 0.0002 | - | - | - | - |
1.5492 | 1220 | 0.004 | - | - | - | - |
1.5619 | 1230 | 0.0017 | - | - | - | - |
1.5746 | 1240 | 0.0003 | - | - | - | - |
1.5873 | 1250 | 0.0027 | - | - | - | - |
1.6 | 1260 | 0.1134 | - | - | - | - |
1.6127 | 1270 | 0.0007 | - | - | - | - |
1.6254 | 1280 | 0.0005 | - | - | - | - |
1.6381 | 1290 | 0.0008 | - | - | - | - |
1.6508 | 1300 | 0.0001 | - | - | - | - |
1.6635 | 1310 | 0.0023 | - | - | - | - |
1.6762 | 1320 | 0.0005 | - | - | - | - |
1.6889 | 1330 | 0.0004 | - | - | - | - |
1.7016 | 1340 | 0.0003 | - | - | - | - |
1.7143 | 1350 | 0.0347 | - | - | - | - |
1.7270 | 1360 | 0.0339 | - | - | - | - |
1.7397 | 1370 | 0.0003 | - | - | - | - |
1.7524 | 1380 | 0.0005 | - | - | - | - |
1.7651 | 1390 | 0.0002 | - | - | - | - |
1.7778 | 1400 | 0.0031 | - | - | - | - |
1.7905 | 1410 | 0.0002 | - | - | - | - |
1.8032 | 1420 | 0.0012 | - | - | - | - |
1.8159 | 1430 | 0.0002 | - | - | - | - |
1.8286 | 1440 | 0.0002 | - | - | - | - |
1.8413 | 1450 | 0.0004 | - | - | - | - |
1.8540 | 1460 | 0.011 | - | - | - | - |
1.8667 | 1470 | 0.0824 | - | - | - | - |
1.8794 | 1480 | 0.0003 | - | - | - | - |
1.8921 | 1490 | 0.0004 | - | - | - | - |
1.9048 | 1500 | 0.0006 | - | - | - | - |
1.9175 | 1510 | 0.015 | - | - | - | - |
1.9302 | 1520 | 0.0004 | - | - | - | - |
1.9429 | 1530 | 0.0004 | - | - | - | - |
1.9556 | 1540 | 0.0011 | - | - | - | - |
1.9683 | 1550 | 0.0003 | - | - | - | - |
1.9810 | 1560 | 0.0006 | - | - | - | - |
1.9937 | 1570 | 0.0042 | - | - | - | - |
2.0 | 1575 | - | 0.7862 | 0.7855 | 0.7852 | 0.7878 |
2.0063 | 1580 | 0.0005 | - | - | - | - |
2.0190 | 1590 | 0.002 | - | - | - | - |
2.0317 | 1600 | 0.0013 | - | - | - | - |
2.0444 | 1610 | 0.0002 | - | - | - | - |
2.0571 | 1620 | 0.0035 | - | - | - | - |
2.0698 | 1630 | 0.0004 | - | - | - | - |
2.0825 | 1640 | 0.0002 | - | - | - | - |
2.0952 | 1650 | 0.0032 | - | - | - | - |
2.1079 | 1660 | 0.0916 | - | - | - | - |
2.1206 | 1670 | 0.0002 | - | - | - | - |
2.1333 | 1680 | 0.0006 | - | - | - | - |
2.1460 | 1690 | 0.0002 | - | - | - | - |
2.1587 | 1700 | 0.0003 | - | - | - | - |
2.1714 | 1710 | 0.0001 | - | - | - | - |
2.1841 | 1720 | 0.0001 | - | - | - | - |
2.1968 | 1730 | 0.0004 | - | - | - | - |
2.2095 | 1740 | 0.0004 | - | - | - | - |
2.2222 | 1750 | 0.0001 | - | - | - | - |
2.2349 | 1760 | 0.0002 | - | - | - | - |
2.2476 | 1770 | 0.0007 | - | - | - | - |
2.2603 | 1780 | 0.0001 | - | - | - | - |
2.2730 | 1790 | 0.0002 | - | - | - | - |
2.2857 | 1800 | 0.0004 | - | - | - | - |
2.2984 | 1810 | 0.0711 | - | - | - | - |
2.3111 | 1820 | 0.0001 | - | - | - | - |
2.3238 | 1830 | 0.0005 | - | - | - | - |
2.3365 | 1840 | 0.0004 | - | - | - | - |
2.3492 | 1850 | 0.0001 | - | - | - | - |
2.3619 | 1860 | 0.0005 | - | - | - | - |
2.3746 | 1870 | 0.0003 | - | - | - | - |
2.3873 | 1880 | 0.0001 | - | - | - | - |
2.4 | 1890 | 0.0002 | - | - | - | - |
2.4127 | 1900 | 0.0001 | - | - | - | - |
2.4254 | 1910 | 0.0002 | - | - | - | - |
2.4381 | 1920 | 0.0002 | - | - | - | - |
2.4508 | 1930 | 0.0002 | - | - | - | - |
2.4635 | 1940 | 0.0004 | - | - | - | - |
2.4762 | 1950 | 0.0001 | - | - | - | - |
2.4889 | 1960 | 0.0002 | - | - | - | - |
2.5016 | 1970 | 0.0002 | - | - | - | - |
2.5143 | 1980 | 0.0001 | - | - | - | - |
2.5270 | 1990 | 0.0001 | - | - | - | - |
2.5397 | 2000 | 0.0002 | - | - | - | - |
2.5524 | 2010 | 0.0023 | - | - | - | - |
2.5651 | 2020 | 0.0002 | - | - | - | - |
2.5778 | 2030 | 0.0001 | - | - | - | - |
2.5905 | 2040 | 0.0003 | - | - | - | - |
2.6032 | 2050 | 0.0003 | - | - | - | - |
2.6159 | 2060 | 0.0002 | - | - | - | - |
2.6286 | 2070 | 0.0001 | - | - | - | - |
2.6413 | 2080 | 0.0 | - | - | - | - |
2.6540 | 2090 | 0.0001 | - | - | - | - |
2.6667 | 2100 | 0.0001 | - | - | - | - |
2.6794 | 2110 | 0.0001 | - | - | - | - |
2.6921 | 2120 | 0.0001 | - | - | - | - |
2.7048 | 2130 | 0.0001 | - | - | - | - |
2.7175 | 2140 | 0.0048 | - | - | - | - |
2.7302 | 2150 | 0.0005 | - | - | - | - |
2.7429 | 2160 | 0.0001 | - | - | - | - |
2.7556 | 2170 | 0.0001 | - | - | - | - |
2.7683 | 2180 | 0.0001 | - | - | - | - |
2.7810 | 2190 | 0.0001 | - | - | - | - |
2.7937 | 2200 | 0.0001 | - | - | - | - |
2.8063 | 2210 | 0.0001 | - | - | - | - |
2.8190 | 2220 | 0.0001 | - | - | - | - |
2.8317 | 2230 | 0.0002 | - | - | - | - |
2.8444 | 2240 | 0.0036 | - | - | - | - |
2.8571 | 2250 | 0.0001 | - | - | - | - |
2.8698 | 2260 | 0.0368 | - | - | - | - |
2.8825 | 2270 | 0.0003 | - | - | - | - |
2.8952 | 2280 | 0.0002 | - | - | - | - |
2.9079 | 2290 | 0.0001 | - | - | - | - |
2.9206 | 2300 | 0.0005 | - | - | - | - |
2.9333 | 2310 | 0.0001 | - | - | - | - |
2.9460 | 2320 | 0.0001 | - | - | - | - |
2.9587 | 2330 | 0.0003 | - | - | - | - |
2.9714 | 2340 | 0.0001 | - | - | - | - |
2.9841 | 2350 | 0.0001 | - | - | - | - |
2.9968 | 2360 | 0.0002 | - | - | - | - |
2.9994 | 2362 | - | 0.7864 | 0.7805 | 0.7838 | 0.7852 |
3.0095 | 2370 | 0.0025 | - | - | - | - |
3.0222 | 2380 | 0.0002 | - | - | - | - |
3.0349 | 2390 | 0.0001 | - | - | - | - |
3.0476 | 2400 | 0.0001 | - | - | - | - |
3.0603 | 2410 | 0.0001 | - | - | - | - |
3.0730 | 2420 | 0.0001 | - | - | - | - |
3.0857 | 2430 | 0.0001 | - | - | - | - |
3.0984 | 2440 | 0.0002 | - | - | - | - |
3.1111 | 2450 | 0.0116 | - | - | - | - |
3.1238 | 2460 | 0.0002 | - | - | - | - |
3.1365 | 2470 | 0.0001 | - | - | - | - |
3.1492 | 2480 | 0.0001 | - | - | - | - |
3.1619 | 2490 | 0.0001 | - | - | - | - |
3.1746 | 2500 | 0.0001 | - | - | - | - |
3.1873 | 2510 | 0.0001 | - | - | - | - |
3.2 | 2520 | 0.0001 | - | - | - | - |
3.2127 | 2530 | 0.0001 | - | - | - | - |
3.2254 | 2540 | 0.0001 | - | - | - | - |
3.2381 | 2550 | 0.0002 | - | - | - | - |
3.2508 | 2560 | 0.0001 | - | - | - | - |
3.2635 | 2570 | 0.0001 | - | - | - | - |
3.2762 | 2580 | 0.0001 | - | - | - | - |
3.2889 | 2590 | 0.0001 | - | - | - | - |
3.3016 | 2600 | 0.063 | - | - | - | - |
3.3143 | 2610 | 0.0001 | - | - | - | - |
3.3270 | 2620 | 0.0001 | - | - | - | - |
3.3397 | 2630 | 0.0001 | - | - | - | - |
3.3524 | 2640 | 0.0001 | - | - | - | - |
3.3651 | 2650 | 0.0002 | - | - | - | - |
3.3778 | 2660 | 0.0001 | - | - | - | - |
3.3905 | 2670 | 0.0001 | - | - | - | - |
3.4032 | 2680 | 0.0001 | - | - | - | - |
3.4159 | 2690 | 0.0001 | - | - | - | - |
3.4286 | 2700 | 0.0001 | - | - | - | - |
3.4413 | 2710 | 0.0001 | - | - | - | - |
3.4540 | 2720 | 0.0002 | - | - | - | - |
3.4667 | 2730 | 0.0001 | - | - | - | - |
3.4794 | 2740 | 0.0001 | - | - | - | - |
3.4921 | 2750 | 0.0001 | - | - | - | - |
3.5048 | 2760 | 0.0001 | - | - | - | - |
3.5175 | 2770 | 0.0002 | - | - | - | - |
3.5302 | 2780 | 0.0001 | - | - | - | - |
3.5429 | 2790 | 0.0001 | - | - | - | - |
3.5556 | 2800 | 0.0001 | - | - | - | - |
3.5683 | 2810 | 0.0001 | - | - | - | - |
3.5810 | 2820 | 0.0001 | - | - | - | - |
3.5937 | 2830 | 0.0001 | - | - | - | - |
3.6063 | 2840 | 0.0001 | - | - | - | - |
3.6190 | 2850 | 0.0 | - | - | - | - |
3.6317 | 2860 | 0.0001 | - | - | - | - |
3.6444 | 2870 | 0.0001 | - | - | - | - |
3.6571 | 2880 | 0.0001 | - | - | - | - |
3.6698 | 2890 | 0.0001 | - | - | - | - |
3.6825 | 2900 | 0.0001 | - | - | - | - |
3.6952 | 2910 | 0.0001 | - | - | - | - |
3.7079 | 2920 | 0.0001 | - | - | - | - |
3.7206 | 2930 | 0.0003 | - | - | - | - |
3.7333 | 2940 | 0.0001 | - | - | - | - |
3.7460 | 2950 | 0.0001 | - | - | - | - |
3.7587 | 2960 | 0.0001 | - | - | - | - |
3.7714 | 2970 | 0.0002 | - | - | - | - |
3.7841 | 2980 | 0.0001 | - | - | - | - |
3.7968 | 2990 | 0.0001 | - | - | - | - |
3.8095 | 3000 | 0.0001 | - | - | - | - |
3.8222 | 3010 | 0.0001 | - | - | - | - |
3.8349 | 3020 | 0.0002 | - | - | - | - |
3.8476 | 3030 | 0.0001 | - | - | - | - |
3.8603 | 3040 | 0.0001 | - | - | - | - |
3.8730 | 3050 | 0.0214 | - | - | - | - |
3.8857 | 3060 | 0.0001 | - | - | - | - |
3.8984 | 3070 | 0.0001 | - | - | - | - |
3.9111 | 3080 | 0.0001 | - | - | - | - |
3.9238 | 3090 | 0.0001 | - | - | - | - |
3.9365 | 3100 | 0.0001 | - | - | - | - |
3.9492 | 3110 | 0.0001 | - | - | - | - |
3.9619 | 3120 | 0.0001 | - | - | - | - |
3.9746 | 3130 | 0.0001 | - | - | - | - |
3.9873 | 3140 | 0.0001 | - | - | - | - |
3.9975 | 3148 | - | 0.7867 | 0.7838 | 0.7827 | 0.7843 |
- 加粗行表示保存的检查点。
📄 许可证
本模型使用 apache-2.0 许可证。
📚 引用
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MatryoshkaLoss
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Jina Embeddings V3
Jina Embeddings V3 是一个多语言句子嵌入模型,支持超过100种语言,专注于句子相似度和特征提取任务。
文本嵌入
Transformers 支持多种语言

J
jinaai
3.7M
911
Ms Marco MiniLM L6 V2
Apache-2.0
基于MS Marco段落排序任务训练的交叉编码器模型,用于信息检索中的查询-段落相关性评分
文本嵌入 英语
M
cross-encoder
2.5M
86
Opensearch Neural Sparse Encoding Doc V2 Distill
Apache-2.0
基于蒸馏技术的稀疏检索模型,专为OpenSearch优化,支持免推理文档编码,在搜索相关性和效率上优于V1版本
文本嵌入
Transformers 英语

O
opensearch-project
1.8M
7
Sapbert From PubMedBERT Fulltext
Apache-2.0
基于PubMedBERT的生物医学实体表征模型,通过自对齐预训练优化语义关系捕捉
文本嵌入 英语
S
cambridgeltl
1.7M
49
Gte Large
MIT
GTE-Large 是一个强大的句子转换器模型,专注于句子相似度和文本嵌入任务,在多个基准测试中表现出色。
文本嵌入 英语
G
thenlper
1.5M
278
Gte Base En V1.5
Apache-2.0
GTE-base-en-v1.5 是一个英文句子转换器模型,专注于句子相似度任务,在多个文本嵌入基准测试中表现优异。
文本嵌入
Transformers 支持多种语言

G
Alibaba-NLP
1.5M
63
Gte Multilingual Base
Apache-2.0
GTE Multilingual Base 是一个多语言的句子嵌入模型,支持超过50种语言,适用于句子相似度计算等任务。
文本嵌入
Transformers 支持多种语言

G
Alibaba-NLP
1.2M
246
Polybert
polyBERT是一个化学语言模型,旨在实现完全由机器驱动的超快聚合物信息学。它将PSMILES字符串映射为600维密集指纹,以数值形式表示聚合物化学结构。
文本嵌入
Transformers

P
kuelumbus
1.0M
5
Bert Base Turkish Cased Mean Nli Stsb Tr
Apache-2.0
基于土耳其语BERT的句子嵌入模型,专为语义相似度任务优化
文本嵌入
Transformers 其他

B
emrecan
1.0M
40
GIST Small Embedding V0
MIT
基于BAAI/bge-small-en-v1.5模型微调的文本嵌入模型,通过MEDI数据集与MTEB分类任务数据集训练,优化了检索任务的查询编码能力。
文本嵌入
Safetensors 英语
G
avsolatorio
945.68k
29
精选推荐AI模型
Llama 3 Typhoon V1.5x 8b Instruct
专为泰语设计的80亿参数指令模型,性能媲美GPT-3.5-turbo,优化了应用场景、检索增强生成、受限生成和推理任务
大型语言模型
Transformers 支持多种语言

L
scb10x
3,269
16
Cadet Tiny
Openrail
Cadet-Tiny是一个基于SODA数据集训练的超小型对话模型,专为边缘设备推理设计,体积仅为Cosmo-3B模型的2%左右。
对话系统
Transformers 英语

C
ToddGoldfarb
2,691
6
Roberta Base Chinese Extractive Qa
基于RoBERTa架构的中文抽取式问答模型,适用于从给定文本中提取答案的任务。
问答系统 中文
R
uer
2,694
98