🚀 SGPT-1.3B加权平均-msmarco-specb-bitfit
该模型主要用于句子特征提取和相似度计算，在多个分类、检索、聚类等任务中有着具体的表现，可应用于文本分类、信息检索、文本聚类等自然语言处理场景。
📚 详细文档
标签信息
| 属性 |
详情 |
| 标签 |
sentence-transformers、feature-extraction、sentence-similarity、mteb |
模型评估结果
模型名为 SGPT-1.3B-weightedmean-msmarco-specb-bitfit，在多个任务和数据集上进行了评估，以下是具体结果：
分类任务
| 数据集类型 |
数据集名称 |
配置 |
分割 |
准确率 |
AP |
F1 |
| mteb/amazon_counterfactual |
MTEB AmazonCounterfactualClassification (en) |
en |
test |
65.20895522388061 |
29.59212705444778 |
59.97099864321921 |
| mteb/amazon_polarity |
MTEB AmazonPolarityClassification |
default |
test |
73.20565 |
67.36680643550963 |
72.90420520325125 |
| mteb/amazon_reviews_multi |
MTEB AmazonReviewsClassification (en) |
en |
test |
34.955999999999996 |
- |
34.719324437696955 |
| mteb/banking77 |
MTEB Banking77Classification |
default |
test |
82.05844155844156 |
- |
82.0185837884764 |
检索任务
| 数据集类型 | 数据集名称 | 配置 | 分割 | MAP@1 | MAP@10 | MAP@100 | MAP@1000 | MRR@1 | MRR@10 | MRR@100 | MRR@1000 | NDCG@1 | NDCG@10 | NDCG@100 | NDCG@1000 | 准确率@1 | 准确率@10 | 准确率@100 | 准确率@1000 | 召回率@1 | 召回率@10 | 召回率@100 | 召回率@1000 | 召回率@3 | 召回率@5 |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| arguana | MTEB ArguAna | default | test | 26.101999999999997 | 40.958 | 42.033 | 42.042 | 26.387 | 41.051 | 42.118 | 42.126999999999995 | 26.101999999999997 | 49.68 | 54.257999999999996 | 54.486000000000004 | 26.101999999999997 | 7.781000000000001 | 0.979 | 0.1 | 26.101999999999997 | 77.809 | 97.866 | 99.644 | 50.141999999999996 | 60.171 |
| BeIR/cqadupstack | MTEB CQADupstackAndroidRetrieval | default | test | 26.519 | 35.634 | 36.961 | 37.088 | 32.332 | 41.168 | 41.977 | 42.028999999999996 | 32.332 | 41.471000000000004 | 46.955999999999996 | 49.262 | 32.332 | 7.7829999999999995 | 1.29 | 0.178 | 26.519 | 53.190000000000005 | 76.56500000000001 | 91.47800000000001 | 38.034 | 45.245999999999995 |
| BeIR/cqadupstack | MTEB CQADupstackEnglishRetrieval | default | test | 25.356 | 34.596 | 35.714 | 35.839999999999996 | 31.274 | 39.592 | 40.284 | 40.339999999999996 | 31.274 | 39.766 | 44.028 | 46.445 | 31.274 | 7.452 | 1.217 | 0.16999999999999998 | 25.356 | 49.344 | 67.497 | 83.372 | 38.227 | 43.187999999999995 |
| BeIR/cqadupstack | MTEB CQADupstackGamingRetrieval | default | test | 32.759 | 43.937 | 45.004 | 45.07 | 37.367 | 47.237 | 47.973 | 48.010999999999996 | 37.367 | 49.659 | 54.069 | 55.552 | 37.367 | 8.163 | 1.133 | 0.131 | 32.759 | 63.341 | 82.502 | 93.259 | 48.796 | 54.921 |
| BeIR/cqadupstack | MTEB CQADupstackGisRetrieval | default | test | 18.962 | 25.863000000000003 | 26.817999999999998 | 26.918 | 20.452 | 27.301 | 28.233000000000004 | 28.310000000000002 | 20.452 | 30.354999999999997 | 35.336 | 37.927 | 20.452 | 4.949 | 0.7799999999999999 | 0.104 | 18.962 | 43.056 | 66.27300000000001 | 85.96000000000001 | 27.776 | 34.287 |
| BeIR/cqadupstack | MTEB CQADupstackMathematicaRetrieval | default | test | 11.24 | 18.503 | 19.553 | 19.689999999999998 | 13.806 | 21.939 | 22.827 | 22.911 | 13.806 | 23.383000000000003 | 28.834 | 32.175 | 13.806 | 4.714 | 0.864 | 0.13 | 11.24 | 34.854 | 59.50299999999999 | 83.25 | 22.02 | 26.715 |
| BeIR/cqadupstack | MTEB CQADupstackPhysicsRetrieval | default | test | 23.012 | 33.048 | 34.371 | 34.489 | 28.104000000000003 | 37.99 | 38.836 | 38.891 | 28.104000000000003 | 39.037 | 44.643 | 46.939 | 28.104000000000003 | 7.2669999999999995 | 1.193 | 0.159 | 23.012 | 52.054 | 75.622 | 90.675 | 37.282 | 43.307 |
| BeIR/cqadupstack | MTEB CQADupstackProgrammersRetrieval | default | test | 21.624 | 30.209999999999997 | 31.52 | 31.625999999999998 | 26.941 | 35.13 | 36.15 | 36.204 | 26.941 | 35.726 | 41.725 | 44.105 | 26.941 | 6.654999999999999 | 1.1520000000000001 | 0.152 | 21.624 | 47.359 | 73.436 | 89.988 | 32.34 | 39.856 |
| BeIR/cqadupstack | MTEB CQADupstackRetrieval | default | test | 20.67566666666667 | 28.479333333333333 | 29.612249999999996 | 29.731166666666663 | 24.402583333333332 | 32.07041666666667 | 32.95841666666667 | 33.025416666666665 | 24.402583333333332 | 33.326166666666666 | 38.51566666666667 | 41.13791666666667 | 24.402583333333332 | 5.943749999999999 | 1.0098333333333334 | 0.14183333333333334 | 20.67566666666667 | 44.245583333333336 | 67.31116666666667 | 85.87841666666665 | 31.49258333333333 | 36.93241666666667 |
| BeIR/cqadupstack | MTEB CQADupstackStatsRetrieval | default | test | 18.34 | 23.988 | 24.895 | 24.992 | 20.399 | 26.186 | 27.017999999999997 | 27.090999999999998 | 20.399 | 27.799000000000003 | 32.579 | 35.209 | 20.399 | 4.585999999999999 | 0.755 | 0.105 | 18.34 | 37.456 | - | - | - | - |
聚类任务
| 数据集类型 |
数据集名称 |
配置 |
分割 |
V-measure |
| mteb/arxiv-clustering-p2p |
MTEB ArxivClusteringP2P |
default |
test |
43.384194916953774 |
| mteb/arxiv-clustering-s2s |
MTEB ArxivClusteringS2S |
default |
test |
33.70962633433912 |
| mteb/biorxiv-clustering-p2p |
MTEB BiorxivClusteringP2P |
default |
test |
35.05918333141837 |
| mteb/biorxiv-clustering-s2s |
MTEB BiorxivClusteringS2S |
default |
test |
30.71055028830579 |
重排序任务
| 数据集类型 |
数据集名称 |
配置 |
分割 |
MAP |
MRR |
| mteb/askubuntudupquestions-reranking |
MTEB AskUbuntuDupQuestions |
default |
test |
58.133058996870076 |
72.10922041946972 |
语义文本相似度任务
| 数据集类型 |
数据集名称 |
配置 |
分割 |
余弦相似度皮尔逊相关系数 |
余弦相似度斯皮尔曼相关系数 |
欧几里得距离皮尔逊相关系数 |
欧几里得距离斯皮尔曼相关系数 |
曼哈顿距离皮尔逊相关系数 |
曼哈顿距离斯皮尔曼相关系数 |
| mteb/biosses-sts |
MTEB BIOSSES |
default |
test |
86.62153841660047 |
83.01514456843276 |
86.00431518427241 |
83.85552516285783 |
85.83025803351181 |
83.86636878343106 |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Transformers 支持多种语言
Safetensors 英语