🚀 Peft
Peft is a library related to sentence similarity, offering various text - related capabilities such as text embedding, information retrieval, and text classification.
📚 Documentation
Model Information
Property |
Details |
Library Name |
peft |
Pipeline Tag |
sentence - similarity |
Tags |
text - embedding, embeddings, information - retrieval, beir, text - classification, language - model, text - clustering, text - semantic - similarity, text - evaluation, text - reranking, feature - extraction, sentence - similarity, Sentence Similarity, natural_questions, ms_marco, fever, hotpot_qa, mteb |
Model Results
The following are the results of the model named "LLM2Vec - Sheared - LLaMA - supervised" on different tasks and datasets:
Classification Tasks
Task |
Dataset |
Config |
Split |
Revision |
Accuracy |
AP |
F1 |
Classification |
mteb/amazon_counterfactual |
en |
test |
e8379541af4e31359cca9fbcf4b00f2671dba205 |
77.41791044776119 |
41.45458580415683 |
71.63305447032735 |
Classification |
mteb/amazon_polarity |
default |
test |
e2d317d38cd51312af73b3d32a06d1a08b442046 |
82.0527 |
77.3222852456055 |
81.97981459031165 |
Classification |
mteb/amazon_reviews_multi |
en |
test |
1399c76144fd37290681b995c656ef9b2e06e26d |
40.806000000000004 |
- |
40.3299129176701 |
Classification |
mteb/banking77 |
default |
test |
0fd18e25b25c072e09e0d92ab615fda904d66300 |
86.00974025974025 |
- |
85.97349359388288 |
Retrieval Tasks
| Task | Dataset | Config | Split | Revision | MAP@1 | MAP@10 | MAP@100 | MAP@1000 | MAP@3 | MAP@5 | MRR@1 | MRR@10 | MRR@100 | MRR@1000 | MRR@3 | MRR@5 | NDCG@1 | NDCG@10 | NDCG@100 | NDCG@1000 | NDCG@3 | NDCG@5 | Precision@1 | Precision@10 | Precision@100 | Precision@1000 | Precision@3 | Precision@5 | Recall@1 | Recall@10 | Recall@100 | Recall@1000 | Recall@3 | Recall@5 |
|------|---------|--------|-------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|
| Retrieval | arguana | default | test | None | 25.391000000000002 | 41.919000000000004 | 42.846000000000004 | 42.851 | 36.260999999999996 | 39.528999999999996 | 26.245 | 42.215 | 43.135 | 43.14 | 36.546 | 39.782000000000004 | 25.391000000000002 | 51.663000000000004 | 55.419 | 55.517 | 39.96 | 45.909 | 25.391000000000002 | 8.3 | 0.989 | 0.1 | 16.904 | 13.058 | 25.391000000000002 | 83.001 | 98.933 | 99.644 | 50.711 | 65.292 |
| Retrieval | cqadupstack/android | default | test | None | 33.043 | 44.515 | 45.967999999999996 | 46.098 | 40.285 | 42.841 | 40.2 | 50.233000000000004 | 50.938 | 50.978 | 47.353 | 49.034 | 40.2 | 51.096 | 56.267999999999994 | 58.092999999999996 | 45.09 | 48.198 | 40.2 | 9.843 | 1.546 | 0.20400000000000001 | 21.507 | 15.966 | 33.043 | 63.871 | 85.527 | 96.936 | 46.859 | 55.116 |
| Retrieval | cqadupstack/english | default | test | None | 31.924000000000003 | 42.298 | 43.589 | 43.724000000000004 | 39.739999999999995 | 41.131 | 40.064 | 48.4 | 49.07 | 49.113 | 46.635 | 47.549 | 40.064 | 47.686 | 52.054 | 54.151 | 44.57 | 45.727000000000004 | 40.064 | 8.770999999999999 | 1.422 | 0.19 | 21.741 | 14.790000000000001 | 31.924000000000003 | 56.603 | 74.82900000000001 | 88.176 | 46.11 | 50.273999999999994 |
| Retrieval | cqadupstack/gaming | default | test | None | 40.721000000000004 | 53.053 | 54.103 | 54.157999999999994 | 49.854 | 51.547 | 46.833999999999996 | 56.61000000000001 | 57.286 | 57.312 | 54.17999999999999 | 55.503 | 46.833999999999996 | 58.928000000000004 | 62.939 | 63.970000000000006 | 53.599 | 55.96600000000001 | 46.833999999999996 | 9.48 | 1.2349999999999999 | 0.13699999999999998 | 24.032999999999998 | 16.213 | 40.721000000000004 | 72.653 | 89.91900000000001 | 97.092 | 58.135999999999996 | 64.156 |
| Retrieval | cqadupstack/gis | default | test | None | 24.938 | 34.027 | 34.999 | 35.083 | 31.154 | 32.767 | 27.006000000000004 | 36.192 | 36.989 | 37.053999999999995 | 33.503 | 34.977000000000004 | 27.006000000000004 | 39.297 | 44.078 | 46.162 | 33.695 | 36.401 | 27.006000000000004 | 6.181 | 0.905 | 0.11199999999999999 | 14.426 | 10.215 | 24.938 | 53.433 | 75.558 | 91.096 | 38.421 | 44.906 |
| Retrieval | cqadupstack/mathematica | default | test | None | 15.565999999999999 | 23.419999999999998 | 24.678 | 24.801000000000002 | 20.465 | 21.979000000000003 | 19.652 | 27.929 | 28.92 | 28.991 | 25.249 | 26.66 | 19.652 | 28.869 | 34.675 | 37.577 | 23.535 | 25.807999999999996 | 19.652 | 5.659 | 0.979 | 0.13699999999999998 | 11.401 | 8.581999999999999 | 15.565999999999999 | 41.163 | 66.405 | 87.071 | 26.478 | 32.217 |
| Retrieval | cqadupstack/physics | default | test | None | 30.834 | 41.49 | 42.897999999999996 | 43.004 | 38.151 | 40.157 | 38.306000000000004 | 47.371 | 48.265 | 48.304 | 44.915 | 46.516999999999996 | 38.306000000000004 | 47.394999999999996 | 53.086999999999996 | 54.94799999999999 | 42.384 | 45.055 | 38.306000000000004 | 8.624 | 1.325 | 0.165 | 20.18 | 14.418000000000001 | 30.834 | 58.977000000000004 | 82.78 | 94.825 | 44.954 | 51.925 |
| Retrieval | cqadupstack/programmers | default | test | None | 28.549000000000003 | 38.796 | 40.085 | 40.198 | 35.412 | 37.116 | 35.388 | 44.626 | 45.445 | 45.491 | 41.952 | 43.368 | 35.388 | 44.894 | 50.166999999999994 | 52.308 | 39.478 | 41.608000000000004 | 35.388 | 8.322000000000001 | 1.2670000000000001 | 0.164 | 18.836 | 13.333 | 28.549000000000003 | 57.229 | 79.541 | 93.887 | 42.056 | 47.705999999999996 |
| Retrieval | mteb/cqadupstack | default | test | None | 26.897333333333336 | 36.28758333333334 | 37.480083333333326 | 37.59683333333333 | 33.3485 | 34.98283333333334 | 31.98916666666667 | 40.61116666666666 | 41.42133333333333 | 41.476333333333336 | 38.19366666666667 | 39.53125 | 31.98916666666667 | 41.73475 | 46.72291666666666 | 48.94916666666666 | 36.883833333333335 | 39.114 | 31.98916666666667 | 7.364083333333335 | 1.1604166666666667 | 0.15433333333333335 | 17.067500000000003 | 12.091916666666666 | 26.897333333333336 | 53.485749999999996 | 75.38716666666666 | 90.75841666666666 | 39.86725 | 45.683416666666666 |
| Retrieval | cqadupstack/stats | default | test | None | 23.544 | 30.85 | 31.674000000000003 | 31.778000000000002 | 28.451999999999998 | 29.797 | 26.687 | 33.725 | 34.439 | 34.512 | 31.493 | 32.735 | 26.687 | 35.207 | 39.406 | 42.021 | 30.842000000000002 | 32.882 | 26.687 | 5.66 | 0.836 | 0.11299999999999999 | 13.395000000000001 | 9.386999999999999 | 23.544 | 45.769 | 65.33200000000001 | 84.82499999999999 | 33.665 | 38.795 |
Clustering Tasks
Task |
Dataset |
Config |
Split |
Revision |
V - Measure |
Clustering |
mteb/arxiv - clustering - p2p |
default |
test |
a122ad7f3f0291bf49cc6f4d32aa80929df69d5d |
43.472186058302285 |
Clustering |
mteb/arxiv - clustering - s2s |
default |
test |
f910caf1a6075f7329cdf8c1a6135696f37dbd53 |
39.846039374129546 |
Clustering |
mteb/biorxiv - clustering - p2p |
default |
test |
65b79d1d13f80053f67aca9498d9402c2d9f1f40 |
37.102075665637685 |
Clustering |
mteb/biorxiv - clustering - s2s |
default |
test |
258694dd0231531bc1fd9de6ceb52a0853c6d908 |
34.27583239919031 |
Reranking Task
Task |
Dataset |
Config |
Split |
Revision |
MAP |
MRR |
Reranking |
mteb/askubuntudupquestions - reranking |
default |
test |
2000358ca161889fa9c082cb41daa8dcfb161a54 |
60.713811638804174 |
73.38906476718111 |
STS Task
Task |
Dataset |
Config |
Split |
Revision |
Cosine Similarity Spearman |
STS |
mteb/biosses - sts |
default |
test |
d3fb88f8f02e40887cd149695127462bbcf29b4a |
85.88328221005123 |
📄 License
This project is licensed under the MIT license.