🚀 sgpt-bloom-7b1-msmarco
このモデルは、文の類似度を計算するためのモデルで、様々な自然言語処理タスクでの性能を示しています。
📚 詳細ドキュメント
モデル情報
属性 |
詳情 |
パイプラインタグ |
文の類似度 |
タグ |
sentence-transformers、feature-extraction、sentence-similarity、mteb |
モデル名 |
sgpt-bloom-7b1-msmarco |
評価結果
分類タスク
データセット |
設定 |
精度 (Accuracy) |
AP |
F1 |
MTEB AmazonCounterfactualClassification (en) |
en |
68.05970149253731 |
31.640363460776193 |
62.50025574145796 |
MTEB AmazonCounterfactualClassification (de) |
de |
61.34903640256959 |
75.18797161500426 |
59.04772570730417 |
MTEB AmazonCounterfactualClassification (en-ext) |
en-ext |
67.78110944527737 |
19.218916023322706 |
56.24477391445512 |
MTEB AmazonCounterfactualClassification (ja) |
ja |
58.23340471092078 |
13.20222967424681 |
47.511718095460296 |
MTEB AmazonPolarityClassification |
default |
68.97232499999998 |
63.53632885535693 |
68.62038513152868 |
MTEB AmazonReviewsClassification (en) |
en |
33.855999999999995 |
- |
33.43468222830134 |
MTEB AmazonReviewsClassification (de) |
de |
29.697999999999997 |
- |
29.39935388885501 |
MTEB AmazonReviewsClassification (es) |
es |
35.974000000000004 |
- |
35.25910820714383 |
MTEB AmazonReviewsClassification (fr) |
fr |
35.922 |
- |
35.38637028933444 |
MTEB AmazonReviewsClassification (ja) |
ja |
27.636 |
- |
27.178349955978266 |
MTEB AmazonReviewsClassification (zh) |
zh |
32.632 |
- |
32.08014766494587 |
MTEB Banking77Classification |
default |
84.33441558441558 |
- |
84.31653077470322 |
検索タスク
| データセット | MAP@1 | MAP@10 | MAP@100 | MAP@1000 | MAP@3 | MAP@5 | MRR@1 | MRR@10 | MRR@100 | MRR@1000 | MRR@3 | MRR@5 | NDCG@1 | NDCG@10 | NDCG@100 | NDCG@1000 | NDCG@3 | NDCG@5 | Precision@1 | Precision@10 | Precision@100 | Precision@1000 | Precision@3 | Precision@5 | Recall@1 | Recall@10 | Recall@100 | Recall@1000 | Recall@3 | Recall@5 |
|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|
| MTEB ArguAna | 23.684 | 38.507999999999996 | 39.677 | 39.690999999999995 | 33.369 | 36.15 | 24.04 | 38.664 | 39.833 | 39.847 | 33.476 | 36.306 | 23.684 | 47.282000000000004 | 52.215 | 52.551 | 36.628 | 41.653 | 23.684 | 7.553 | 0.97 | 0.1 | 15.363 | 11.664 | 23.684 | 75.533 | 97.013 | 99.57300000000001 | 46.088 | 58.321 |
| MTEB CQADupstackAndroidRetrieval | 30.203999999999997 | 41.314 | 42.66 | 42.775999999999996 | 37.614999999999995 | 39.643 | 37.482 | 47.075 | 47.845 | 47.887 | 44.635000000000005 | 45.966 | 37.482 | 47.676 | 52.915 | 54.82900000000001 | 42.562 | 44.852 | 37.482 | 9.142 | 1.436 | 0.189 | 20.458000000000002 | 14.821000000000002 | 30.203999999999997 | 60.343 | 82.58 | 94.813 | 45.389 | 51.800999999999995 |
| MTEB CQADupstackEnglishRetrieval | 30.889 | 40.949999999999996 | 42.131 | 42.253 | 38.346999999999994 | 39.782000000000004 | 38.79 | 46.944 | 47.61 | 47.650999999999996 | 45.053 | 46.101 | 38.79 | 46.286 | 50.637 | 52.649 | 42.851 | 44.311 | 38.79 | 8.516 | 1.3679999999999999 | 0.183 | 20.637 | 14.318 | 30.889 | 55.327000000000005 | 74.091 | 86.75500000000001 | 44.557 | 49.064 |
| MTEB CQADupstackGamingRetrieval | 39.105000000000004 | 50.928 | 51.958000000000006 | 52.017 | 47.638999999999996 | 49.624 | 44.639 | 54.261 | 54.913999999999994 | 54.945 | 51.681999999999995 | 53.290000000000006 | 44.639 | 56.678 | 60.649 | 61.855000000000004 | 51.092999999999996 | 54.096999999999994 | 44.639 | 9.028 | 1.194 | 0.135 | 22.508 | 15.661 | 39.105000000000004 | 70.367 | 87.359 | 95.88 | 55.581 | 62.821000000000005 |
| MTEB CQADupstackGisRetrieval | 23.7 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
クラスタリングタスク
データセット |
V-Measure |
MTEB ArxivClusteringP2P |
44.59375023881131 |
MTEB ArxivClusteringS2S |
38.02921907752556 |
MTEB BiorxivClusteringP2P |
36.025318694698086 |
MTEB BiorxivClusteringS2S |
32.484889034590346 |
再ランキングタスク
データセット |
MAP |
MRR |
MTEB AskUbuntuDupQuestions |
59.97321570342109 |
73.18284746955106 |
STSタスク
データセット |
Cosine Similarity (Pearson) |
Cosine Similarity (Spearman) |
Euclidean (Pearson) |
Euclidean (Spearman) |
Manhattan (Pearson) |
Manhattan (Spearman) |
MTEB BIOSSES |
89.09091435741429 |
85.31459455332202 |
79.3587681410798 |
76.8174129874685 |
79.57051762121769 |
76.75837549768094 |
バイテキストマイニングタスク
データセット |
設定 |
精度 (Accuracy) |
F1 |
Precision |
Recall |
MTEB BUCC (de-en) |
de-en |
54.27974947807933 |
54.00144411132214 |
53.87119374071357 |
54.27974947807933 |
MTEB BUCC (fr-en) |
fr-en |
97.3365617433414 |
97.06141316310809 |
96.92567319685965 |
97.3365617433414 |
MTEB BUCC (ru-en) |
ru-en |
46.05472809144441 |
45.30319274690595 |
45.00015469655234 |
46.05472809144441 |
MTEB BUCC (zh-en) |
zh-en |
98.10426540284361 |
97.96384061786905 |
97.89362822538178 |
98.10426540284361 |