🚀 multilingual-e5-small
multilingual-e5-small
は、多言語テキスト処理タスクで優れた性能を発揮するモデルです。このモデルは、分類、検索、クラスタリング、再ランキング、意味的テキスト類似度など、様々なタイプのタスクとデータセットでテストされており、多言語テキスト処理の包括的な性能評価を提供します。
📚 ドキュメント
モデルタグ
- mteb
- Sentence Transformers
- sentence-similarity
- sentence-transformers
モデル評価結果
分類タスク
データセット |
設定 |
正解率 |
AP |
F1 |
MTEB AmazonCounterfactualClassification (en) |
en |
73.79104477611939 |
36.9996434842022 |
67.95453679103099 |
MTEB AmazonCounterfactualClassification (de) |
de |
71.64882226980728 |
82.11942130026586 |
69.87963421606715 |
MTEB AmazonCounterfactualClassification (en-ext) |
en-ext |
75.8095952023988 |
24.46869495579561 |
63.00108480037597 |
MTEB AmazonCounterfactualClassification (ja) |
ja |
64.186295503212 |
15.496804690197042 |
52.07153895475031 |
MTEB AmazonPolarityClassification |
default |
88.699325 |
85.27039559917269 |
88.65556295032513 |
MTEB AmazonReviewsClassification (en) |
en |
44.69799999999999 |
- |
43.73187348654165 |
MTEB AmazonReviewsClassification (de) |
de |
40.245999999999995 |
- |
39.3863530637684 |
MTEB AmazonReviewsClassification (es) |
es |
40.394 |
- |
39.301223469483446 |
MTEB AmazonReviewsClassification (fr) |
fr |
38.864 |
- |
37.97974261868003 |
MTEB AmazonReviewsClassification (ja) |
ja |
37.682 |
- |
37.07399369768313 |
MTEB AmazonReviewsClassification (zh) |
zh |
37.504 |
- |
36.62317273874278 |
MTEB Banking77Classification |
default |
79.41558441558442 |
- |
79.25886487487219 |
MTEB EmotionClassification |
default |
42.455 |
- |
37.59462649781862 |
MTEB ImdbClassification |
default |
80. |
- |
- |
検索タスク
| データセット | MAP@1 | MAP@10 | MAP@100 | MAP@1000 | MAP@3 | MAP@5 | MRR@1 | MRR@10 | MRR@100 | MRR@1000 | MRR@3 | MRR@5 | NDCG@1 | NDCG@10 | NDCG@100 | NDCG@1000 | NDCG@3 | NDCG@5 | 精度@1 | 精度@10 | 精度@100 | 精度@1000 | 精度@3 | 精度@5 | 再現率@1 | 再現率@10 | 再現率@100 | 再現率@1000 | 再現率@3 | 再現率@5 |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| MTEB ArguAna | 19.061 | 31.703 | 32.967 | 33.001000000000005 | 27.466 | 29.564 | 19.559 | 31.874999999999996 | 33.146 | 33.18 | 27.667 | 29.74 | 19.061 | 39.062999999999995 | 45.184000000000005 | 46.115 | 30.203000000000003 | 33.953 | 19.061 | 6.279999999999999 | 0.9129999999999999 | 0.099 | 12.706999999999999 | 9.431000000000001 | 19.061 | 62.802 | 91.323 | 98.72 | 38.122 | 47.155 |
| MTEB CQADupstackRetrieval | 24.252999999999997 | 31.655916666666666 | 32.680749999999996 | 32.79483333333334 | 29.43691666666666 | 30.717416666666665 | 28.602750000000004 | 35.56875 | 36.3595 | 36.427749999999996 | 33.586166666666664 | 34.73641666666666 | 28.602750000000004 | 36.06933333333334 | 40.70141666666667 | 43.24341666666667 | 32.307916666666664 | 34.129999999999995 | 28.602750000000004 | 6.097666666666667 | 0.9809166666666668 | 0.13766666666666663 | 14.628166666666667 | 10.266916666666667 | 24.252999999999997 | 45.31916666666667 | 66.03575000000001 | 83.94708333333334 | 34.71941666666666 | 39.46358333333333 |
| MTEB ClimateFEVER | 9.024000000000001 | 15.644 | 17.154 | 17.345 | 13.028 | 14.251 | 19.674 | 29.826999999999998 | 30.935000000000002 | 30.987 | 26.645000000000003 | 28.29 | 19.674 | 22.545 | 29.207 | 32.912 | 17.952 | 19.363 | 19.674 | 7.212000000000001 | 1.435 | 0.212 | 13.507 | 10.397 | 9.024000000000001 | 28.077999999999996 | 51.403 | 72.406 | 16.768 | 20.737 |
| MTEB DBPedia | 8.012 | 17.138 | 24.146 | 25.622 | 12.552 | 14.435 | 62.25000000000001 | 71.186 | 71.504 | 71.514 | 69.333 | 70.408 | 49.75 | 37.76 | 42.071 | 49.309 | 41.644 | 39.812999999999995 | 62.25000000000001 | 30.15 | 9.753 | 0.19189999999999998 | 45.667 | 39.15 | 8.012 | 22.599 | 48.068 | 71.328 | 14.043 | 17.124 |
| MTEB FEVER | 58.092 | 69.586 | 69.968 | 69.982 | 67.48100000000001 | 68.915 | 62.166 | 73.588 | 73.86399999999999 | 73.868 | 71.6 | 72.99 | 62.166 | 75.27199999999999 | 76.816 | 77.09700000000001 | 71.36 | 73.785 | 62.166 | 9.716 | 1.065 | 0.11 | 28.278 | 18.343999999999998 | 58.092 | 88.73400000000001 | 95.195 | 97.04599999999999 | 78.45 | 84.316 |
| MTEB FiQA2018 | 16.649 | 26.457000000000004 | 28.169 | 28.352 | 23.305 | 25.169000000000004 | 32.407000000000004 | 40.922 | 41.931000000000004 | 41.983 | 38.786 | 40.205999999999996 | 32.407000000000004 | 33.314 | 40.312 | 43.685 | 30.391000000000002 | 31.525 | 32.407000000000004 | 8.966000000000001 | 1.6019999999999999 | 0.22200000000000003 | 20.165 | 14.722 | 16.649 | 39.117000000000004 | 65.726 | 85.784 | 27.914 | 33.289 |
| MTEB HotpotQA | 36.253 | 56.16799999999999 | 57.06099999999999 | 57.126 | 52.644999999999996 | 54.909 | 72.505 | 79.66 | 79.869 | 79.88 | 78.411 | 79.19800000000001 | 72.505 | 65.094 | 68.219 | 69.515 | 59.99 | 62.909000000000006 | 72.505 | 13.749 | 1.619 | 0.179 | 38.357 | 25.313000000000002 | 36.253 | 68.744 | 80.925 | 89.534 | 57.535000000000004 | 63.282000000000004 |
クラスタリングタスク
データセット |
V-測定値 |
MTEB ArxivClusteringP2P |
39.22266660528253 |
MTEB ArxivClusteringS2S |
30.79980849482483 |
MTEB BiorxivClusteringP2P |
35.747820820329736 |
MTEB BiorxivClusteringS2S |
27.045143830596146 |
再ランキングタスク
データセット |
MAP |
MRR |
MTEB AskUbuntuDupQuestions |
57.8790068352054 |
71.78791276436706 |
意味的テキスト類似度タスク
データセット |
コサイン類似度ピアソン相関係数 |
コサイン類似度スピアマン相関係数 |
ユークリッド距離ピアソン相関係数 |
ユークリッド距離スピアマン相関係数 |
マンハッタン距離ピアソン相関係数 |
マンハッタン距離スピアマン相関係数 |
MTEB BIOSSES |
82.36328364043163 |
82.26211536195868 |
80.3183865039173 |
79.88495276296132 |
80.14484480692127 |
80.39279565980743 |
バイリンガルマイニングタスク
データセット |
正解率 |
F1 |
精度 |
再現率 |
MTEB BUCC (de-en) |
98.0375782881002 |
97.86012526096033 |
97.77139874739039 |
98.0375782881002 |
MTEB BUCC (fr-en) |
93.35241030156286 |
92.66050333846944 |
92.3306919069631 |
93.35241030156286 |
MTEB BUCC (ru-en) |
94.0699688257707 |
93.50236693222492 |
93.22791825424315 |
94.0699688257707 |
MTEB BUCC (zh-en) |
89.25750394944708 |
88.79234684921889 |
88.57293312269616 |
89.25750394944708 |