🚀 cai-lunaris-text-embeddings
This is a model for sentence similarity tasks, which can extract text features and perform retrieval, reranking, and semantic text similarity tasks.
📚 Documentation
Model Information
Property |
Details |
Model Type |
Sentence Similarity Model |
Pipeline Tag |
sentence-similarity |
Tags |
sentence-transformers, feature-extraction, sentence-similarity, transformers, mteb |
Results on Different Datasets
1. ArguAna (Retrieval Task)
Metric |
Value |
map_at_1 |
17.07 |
map_at_10 |
29.372999999999998 |
map_at_100 |
30.79 |
map_at_1000 |
30.819999999999997 |
map_at_3 |
24.395 |
map_at_5 |
27.137 |
mrr_at_1 |
17.923000000000002 |
mrr_at_10 |
29.695 |
mrr_at_100 |
31.098 |
mrr_at_1000 |
31.128 |
mrr_at_3 |
24.704 |
mrr_at_5 |
27.449 |
ndcg_at_1 |
17.07 |
ndcg_at_10 |
37.269000000000005 |
ndcg_at_100 |
43.716 |
ndcg_at_1000 |
44.531 |
ndcg_at_3 |
26.839000000000002 |
ndcg_at_5 |
31.845000000000002 |
precision_at_1 |
17.07 |
precision_at_10 |
6.3020000000000005 |
precision_at_100 |
0.922 |
precision_at_1000 |
0.099 |
precision_at_3 |
11.309 |
precision_at_5 |
9.246 |
recall_at_1 |
17.07 |
recall_at_10 |
63.016000000000005 |
recall_at_100 |
92.24799999999999 |
recall_at_1000 |
98.72 |
recall_at_3 |
33.926 |
recall_at_5 |
46.23 |
2. MTEB AskUbuntuDupQuestions (Reranking Task)
Metric |
Value |
map |
53.44266265900711 |
mrr |
66.54695950402322 |
3. MTEB BIOSSES (STS Task)
Metric |
Value |
cos_sim_pearson |
75.9652953730204 |
cos_sim_spearman |
73.96554077670989 |
euclidean_pearson |
75.68477255792381 |
euclidean_spearman |
74.59447076995703 |
manhattan_pearson |
75.94984623881341 |
manhattan_spearman |
74.72218452337502 |
4. MTEB CQADupstack Datasets (Retrieval Task)
- CQADupstackAndroidRetrieval
Metric |
Value |
map_at_1 |
14.119000000000002 |
map_at_10 |
19.661 |
map_at_100 |
20.706 |
map_at_1000 |
20.848 |
map_at_3 |
17.759 |
map_at_5 |
18.645 |
mrr_at_1 |
17.166999999999998 |
mrr_at_10 |
23.313 |
mrr_at_100 |
24.263 |
mrr_at_1000 |
24.352999999999998 |
mrr_at_3 |
21.412 |
mrr_at_5 |
22.313 |
ndcg_at_1 |
17.166999999999998 |
ndcg_at_10 |
23.631 |
ndcg_at_100 |
28.427000000000003 |
ndcg_at_1000 |
31.862000000000002 |
ndcg_at_3 |
20.175 |
ndcg_at_5 |
21.397 |
precision_at_1 |
17.166999999999998 |
precision_at_10 |
4.549 |
precision_at_100 |
0.8370000000000001 |
precision_at_1000 |
0.136 |
precision_at_3 |
9.68 |
precision_at_5 |
6.981 |
recall_at_1 |
14.119000000000002 |
recall_at_10 |
32.147999999999996 |
recall_at_100 |
52.739999999999995 |
recall_at_1000 |
76.67 |
recall_at_3 |
22.019 |
recall_at_5 |
25.361 |
- CQADupstackEnglishRetrieval
Metric |
Value |
map_at_1 |
16.576 |
map_at_10 |
22.281000000000002 |
map_at_100 |
23.066 |
map_at_1000 |
23.166 |
map_at_3 |
20.385 |
map_at_5 |
21.557000000000002 |
mrr_at_1 |
20.892 |
mrr_at_10 |
26.605 |
mrr_at_100 |
27.229 |
mrr_at_1000 |
27.296 |
mrr_at_3 |
24.809 |
mrr_at_5 |
25.927 |
ndcg_at_1 |
20.892 |
ndcg_at_10 |
26.092 |
ndcg_at_100 |
29.398999999999997 |
ndcg_at_1000 |
31.884 |
ndcg_at_3 |
23.032 |
ndcg_at_5 |
24.634 |
precision_at_1 |
20.892 |
precision_at_10 |
4.885 |
precision_at_100 |
0.818 |
precision_at_1000 |
0.126 |
precision_at_3 |
10.977 |
precision_at_5 |
8.013 |
recall_at_1 |
16.576 |
recall_at_10 |
32.945 |
recall_at_100 |
47.337 |
recall_at_1000 |
64.592 |
recall_at_3 |
24.053 |
recall_at_5 |
28.465 |
- CQADupstackGamingRetrieval
Metric |
Value |
map_at_1 |
20.604 |
map_at_10 |
28.754999999999995 |
map_at_100 |
29.767 |
map_at_1000 |
29.852 |
map_at_3 |
26.268 |
map_at_5 |
27.559 |
mrr_at_1 |
24.326 |
mrr_at_10 |
31.602000000000004 |
mrr_at_100 |
32.46 |
mrr_at_1000 |
32.521 |
mrr_at_3 |
29.415000000000003 |
mrr_at_5 |
30.581000000000003 |
ndcg_at_1 |
24.326 |
ndcg_at_10 |
33.335 |
ndcg_at_100 |
38.086 |
ndcg_at_1000 |
40.319 |
ndcg_at_3 |
28.796 |
ndcg_at_5 |
30.758999999999997 |
precision_at_1 |
24.326 |
precision_at_10 |
5.712 |
precision_at_100 |
0.893 |
precision_at_1000 |
0.11499999999999999 |
precision_at_3 |
13.208 |
precision_at_5 |
9.329 |
recall_at_1 |
20.604 |
recall_at_10 |
44.505 |
recall_at_100 |
65.866 |
recall_at_1000 |
82.61800000000001 |
recall_at_3 |
31.794 |
recall_at_5 |
36.831 |
- CQADupstackGisRetrieval
Metric |
Value |
map_at_1 |
8.280999999999999 |
map_at_10 |
11.636000000000001 |
map_at_100 |
12.363 |
map_at_1000 |
12.469 |
map_at_3 |
10.415000000000001 |
map_at_5 |
11.144 |
mrr_at_1 |
9.266 |
mrr_at_10 |
12.838 |
mrr_at_100 |
13.608999999999998 |
mrr_at_1000 |
13.700999999999999 |
mrr_at_3 |
11.507000000000001 |
mrr_at_5 |
12.343 |
ndcg_at_1 |
9.266 |
ndcg_at_10 |
13.877 |
ndcg_at_100 |
18.119 |
ndcg_at_1000 |
21.247 |
ndcg_at_3 |
11.376999999999999 |
ndcg_at_5 |
12.675 |
precision_at_1 |
9.266 |
precision_at_10 |
2.226 |
precision_at_100 |
0.47200000000000003 |
precision_at_1000 |
0.077 |
precision_at_3 |
4.859 |
precision_at_5 |
3.6380000000000003 |
recall_at_1 |
8.280999999999999 |
recall_at_10 |
19.872999999999998 |
recall_at_100 |
40.585 |
recall_at_1000 |
65.225 |
recall_at_3 |
13.014000000000001 |
recall_at_5 |
16.147 |
- CQADupstackMathematicaRetrieval
Metric |
Value |
map_at_1 |
4.1209999999999996 |
map_at_10 |
7.272 |
map_at_100 |
8.079 |
map_at_1000 |
8.199 |
map_at_3 |
6.212 |
map_at_5 |
6.736000000000001 |
mrr_at_1 |
5.721 |
mrr_at_10 |
9.418 |
mrr_at_100 |
10.281 |
mrr_at_1000 |
10.385 |
mrr_at_3 |
8.126 |
mrr_at_5 |
8.779 |
ndcg_at_1 |
5.721 |
ndcg_at_10 |
9.673 |
ndcg_at_100 |
13.852999999999998 |
ndcg_at_1000 |
17.546999999999997 |
ndcg_at_3 |
7.509 |
ndcg_at_5 |
8.373 |
precision_at_1 |
5.721 |
precision_at_10 |
2.04 |
precision_at_100 |
0.48 |
precision_at_1000 |
0.093 |
precision_at_3 |
4.022 |
precision_at_5 |
3.06 |
recall_at_1 |
4.1209999999999996 |
recall_at_10 |
15.201 |
recall_at_100 |
33.922999999999995 |
recall_at_1000 |
61.529999999999994 |
recall_at_3 |
8.869 |
recall_at_5 |
11.257 |
- CQADupstackPhysicsRetrieval
Metric |
Value |
map_at_1 |
14.09 |
map_at_10 |
19.573999999999998 |
map_at_100 |
20.580000000000002 |
map_at_1000 |
20.704 |
map_at_3 |
17.68 |
map_at_5 |
18.64 |
mrr_at_1 |
17.227999999999998 |
mrr_at_10 |
23.152 |
mrr_at_100 |
24.056 |
mrr_at_1000 |
24.141000000000002 |
mrr_at_3 |
21.142 |
mrr_at_5 |
22.201 |
ndcg_at_1 |
17.227999999999998 |
ndcg_at_10 |
23.39 |
ndcg_at_100 |
28.483999999999998 |
ndcg_at_1000 |
31.709 |
ndcg_at_3 |
19.883 |
ndcg_at_5 |
21.34 |
precision_at_1 |
17.227999999999998 |
precision_at_10 |
4.3790000000000004 |
precision_at_100 |
0.826 |
precision_at_1000 |
0.128 |
precision_at_3 |
9.496 |
precision_at_5 |
6.872 |
recall_at_1 |
14.09 |
recall_at_10 |
31.580000000000002 |
recall_at_100 |
54.074 |
recall_at_1000 |
77.092 |
recall_at_3 |
21.601 |
recall_at_5 |
25.333 |
- CQADupstackProgrammersRetrieval
Metric |
Value |
map_at_1 |
10.538 |
map_at_10 |
15.75 |
map_at_100 |
16.71 |
map_at_1000 |
16.838 |
map_at_3 |
13.488 |
map_at_5 |
14.712 |
mrr_at_1 |
13.813 |
mrr_at_10 |
19.08 |
mrr_at_100 |
19.946 |
mrr_at_1000 |
20.044 |
mrr_at_3 |
16.838 |
mrr_at_5 |
[Value not provided] |
📄 License
This project is licensed under the Apache-2.0 license.