🚀 SGPT-2.7B-weightedmean-msmarco-specb-bitfit
このモデルは文の類似度を計算するためのもので、Sentence Transformersを用いた特徴抽出に特化しています。MTEBの複数のデータセットで評価され、分類、検索、クラスタリング、再ランキング、STSなどのタスクで性能を発揮します。
📚 詳細ドキュメント
モデル情報
| 属性 |
詳情 |
| パイプラインタグ |
文の類似度 |
| タグ |
sentence-transformers、feature-extraction、sentence-similarity、mteb |
| モデル名 |
SGPT-2.7B-weightedmean-msmarco-specb-bitfit |
評価結果
分類タスク
| データセット |
正解率 |
AP |
F1 |
| MTEB AmazonCounterfactualClassification (en) |
67.56716417910448 |
30.75574629595259 |
61.805121301858655 |
| MTEB AmazonPolarityClassification |
71.439575 |
65.91341330532453 |
70.90561852619555 |
| MTEB AmazonReviewsClassification (en) |
35.748000000000005 |
- |
35.48576287186347 |
| MTEB Banking77Classification |
83.21753246753246 |
- |
83.15394543120915 |
検索タスク
| データセット |
MAP@1 |
MAP@10 |
MAP@100 |
MAP@1000 |
MRR@1 |
MRR@10 |
MRR@100 |
MRR@1000 |
NDCG@1 |
NDCG@10 |
NDCG@100 |
NDCG@1000 |
Precision@1 |
Precision@10 |
Precision@100 |
Precision@1000 |
Recall@1 |
Recall@10 |
Recall@100 |
Recall@1000 |
| MTEB ArguAna |
25.96 |
41.619 |
42.673 |
42.684 |
26.316 |
41.772 |
42.82 |
42.83 |
25.96 |
50.491 |
54.864999999999995 |
55.10699999999999 |
25.96 |
7.8950000000000005 |
0.9780000000000001 |
0.1 |
25.96 |
78.947 |
97.795 |
99.644 |
| MTEB CQADupstackAndroidRetrieval |
30.808999999999997 |
40.617 |
41.894999999999996 |
42.025 |
37.482 |
46.497 |
47.144000000000005 |
47.189 |
37.482 |
46.688 |
51.726000000000006 |
53.825 |
37.482 |
8.827 |
1.393 |
0.186 |
30.808999999999997 |
58.47 |
80.51899999999999 |
93.809 |
| MTEB CQADupstackEnglishRetrieval |
26.962000000000003 |
36.93 |
38.102000000000004 |
38.22 |
33.567 |
42.269 |
42.99 |
43.033 |
33.567 |
42.405 |
46.847 |
48.951 |
33.567 |
8.032 |
1.295 |
0.17600000000000002 |
26.962000000000003 |
52.489 |
71.635 |
85.141 |
| MTEB CQADupstackGamingRetrieval |
36.318 |
47.97 |
49.003 |
49.065999999999995 |
41.504999999999995 |
51.431000000000004 |
52.129000000000005 |
52.161 |
41.504999999999995 |
53.676 |
57.867000000000004 |
59.166 |
41.504999999999995 |
8.608 |
1.1560000000000001 |
0.133 |
36.318 |
67.066 |
85.34 |
94.491 |
| MTEB CQADupstackGisRetrieval |
22.167 |
29.543999999999997 |
30.579 |
30.669999999999998 |
24.068 |
31.237 |
32.222 |
32.292 |
24.068 |
33.973 |
39.135 |
41.443999999999996 |
24.068 |
5.299 |
0.823 |
0.106 |
22.167 |
46.115 |
69.867 |
87.234 |
| MTEB CQADupstackMathematicaRetrieval |
12.033000000000001 |
19.314 |
20.562 |
20.695 |
14.801 |
22.74 |
23.876 |
23.949 |
14.801 |
24.038 |
30.186 |
33.321 |
14.801 |
4.776 |
0.897 |
0.133 |
12.033000000000001 |
35.098 |
62.175000000000004 |
84.17099999999999 |
| MTEB CQADupstackPhysicsRetrieval |
26.651000000000003 |
36.901 |
38.249 |
38.361000000000004 |
32.724 |
42.504 |
43.391999999999996 |
43.436 |
32.724 |
43.007 |
48.601 |
50.697 |
32.724 |
7.872999999999999 |
1.247 |
0.16199999999999998 |
26.651000000000003 |
55.674 |
78.904 |
92.55799999999999 |
| MTEB CQADupstackProgrammersRetrieval |
22.589000000000002 |
32.244 |
33.46 |
33.593 |
28.425 |
37.282 |
38.187 |
38.248 |
28.425 |
37.942 |
43.443 |
45.995999999999995 |
28.425 |
7.1 |
1.166 |
0.158 |
22.589000000000002 |
50.03999999999999 |
73.973 |
91.128 |
| MTEB CQADupstackRetrieval |
23.190833333333334 |
31.504916666666666 |
32.64908333333334 |
32.77075 |
27.427499999999995 |
35.36483333333334 |
36.23441666666666 |
36.297583333333336 |
27.427499999999995 |
36.53358333333333 |
41.64508333333333 |
44.14499999999999 |
27.427499999999995 |
6.481083333333333 |
1.0610833333333334 |
0.14691666666666667 |
23.190833333333334 |
47.65175 |
70.41016666666667 |
87.82708333333332 |
| MTEB CQADupstackStatsRetrieval |
20.409 |
26.794 |
27.682000000000002 |
27.783 |
22.853 |
29.296 |
30.103 |
30.179000000000002 |
22.853 |
31.007 |
35.581 |
38.147 |
22.853 |
5.031 |
0.7939999999999999 |
0.11 |
20.409 |
- |
- |
- |
クラスタリングタスク
| データセット |
V-measure |
| MTEB ArxivClusteringP2P |
44.72125714642202 |
| MTEB ArxivClusteringS2S |
35.081451519142064 |
| MTEB BiorxivClusteringP2P |
34.41414219680629 |
| MTEB BiorxivClusteringS2S |
30.533275862270028 |
再ランキングタスク
| データセット | MAP | MRR |
|------|------|
| MTEB AskUbuntuDupQuestions | 59.634661990392054 | 73.6813525040672 |
STSタスク
| データセット |
Cosine Similarity Pearson |
Cosine Similarity Spearman |
Euclidean Pearson |
Euclidean Spearman |
Manhattan Pearson |
Manhattan Spearman |
| MTEB BIOSSES |
87.42754550496836 |
84.84289705838664 |
85.59331970450859 |
85.8525586184271 |
85.41233134466698 |
85.52303303767404 |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Safetensors
Transformers 複数言語対応