🚀 gte-qwen2-7B-instruct
This is a model related to sentence - similarity tasks, integrating technologies such as mteb
, sentence - transformers
, transformers
, and Qwen2
. It has achieved certain performance metrics on multiple datasets.
📚 Documentation
Model Information
Property |
Details |
Model Name |
gte-qwen2-7B-instruct |
Tags |
mteb, sentence-transformers, transformers, Qwen2, sentence-similarity |
Performance Metrics
The model has been tested on various datasets, and the following are the performance metrics:
MTEB AmazonCounterfactualClassification (en)
- Task Type: Classification
- Dataset Revision: e8379541af4e31359cca9fbcf4b00f2671dba205
- Split: test
| Metric Type | Value |
|-------------|-------|
| accuracy | 83.98507462686567 |
| ap | 50.93015252587014 |
| f1 | 78.50416599051215 |
MTEB AmazonPolarityClassification
- Task Type: Classification
- Dataset Revision: e2d317d38cd51312af73b3d32a06d1a08b442046
- Split: test
| Metric Type | Value |
|-------------|-------|
| accuracy | 96.61065 |
| ap | 94.89174052954196 |
| f1 | 96.60942596940565 |
MTEB AmazonReviewsClassification (en)
- Task Type: Classification
- Dataset Revision: 1399c76144fd37290681b995c656ef9b2e06e26d
- Split: test
| Metric Type | Value |
|-------------|-------|
| accuracy | 55.614000000000004 |
| f1 | 54.90553480294904 |
MTEB ArguAna
- Task Type: Retrieval
- Dataset Revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
- Split: test
| Metric Type | Value |
|-------------|-------|
| map_at_1 | 45.164 |
| map_at_10 | 61.519 |
| map_at_100 | 61.769 |
| map_at_1000 | 61.769 |
| map_at_3 | 57.443999999999996 |
| map_at_5 | 60.058 |
| mrr_at_1 | 46.088 |
| mrr_at_10 | 61.861 |
| mrr_at_100 | 62.117999999999995 |
| mrr_at_1000 | 62.117999999999995 |
| mrr_at_3 | 57.729 |
| mrr_at_5 | 60.392 |
| ndcg_at_1 | 45.164 |
| ndcg_at_10 | 69.72 |
| ndcg_at_100 | 70.719 |
| ndcg_at_1000 | 70.719 |
| ndcg_at_3 | 61.517999999999994 |
| ndcg_at_5 | 66.247 |
| precision_at_1 | 45.164 |
| precision_at_10 | 9.545 |
| precision_at_100 | 0.996 |
| precision_at_1000 | 0.1 |
| precision_at_3 | 24.443 |
| precision_at_5 | 16.97 |
| recall_at_1 | 45.164 |
| recall_at_10 | 95.448 |
| recall_at_100 | 99.644 |
| recall_at_1000 | 99.644 |
| recall_at_3 | 73.329 |
| recall_at_5 | 84.851 |
MTEB ArxivClusteringP2P
- Task Type: Clustering
- Dataset Revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
- Split: test
| Metric Type | Value |
|-------------|-------|
| v_measure | 50.511868162026175 |
MTEB ArxivClusteringS2S
- Task Type: Clustering
- Dataset Revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
- Split: test
| Metric Type | Value |
|-------------|-------|
| v_measure | 45.007803189284004 |
MTEB AskUbuntuDupQuestions
- Task Type: Reranking
- Dataset Revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
- Split: test
| Metric Type | Value |
|-------------|-------|
| map | 64.55292107723382 |
| mrr | 77.66158818097877 |
MTEB BIOSSES
- Task Type: STS
- Dataset Revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
- Split: test
| Metric Type | Value |
|-------------|-------|
| cos_sim_pearson | 85.65459047085452 |
| cos_sim_spearman | 82.10729255710761 |
| euclidean_pearson | 82.78079159312476 |
| euclidean_spearman | 80.50002701880933 |
| manhattan_pearson | 82.41372641383016 |
| manhattan_spearman | 80.57412509272639 |
MTEB Banking77Classification
- Task Type: Classification
- Dataset Revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
- Split: test
| Metric Type | Value |
|-------------|-------|
| accuracy | 87.30844155844156 |
| f1 | 87.25307322443255 |
MTEB BiorxivClusteringP2P
- Task Type: Clustering
- Dataset Revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
- Split: test
| Metric Type | Value |
|-------------|-------|
| v_measure | 43.20754608934859 |
MTEB BiorxivClusteringS2S
- Task Type: Clustering
- Dataset Revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
- Split: test
| Metric Type | Value |
|-------------|-------|
| v_measure | 38.818037697335505 |
MTEB CQADupstackAndroidRetrieval
- Task Type: Retrieval
- Dataset Revision: f46a197baaae43b4f621051089b82a364682dfeb
- Split: test
| Metric Type | Value |
|-------------|-------|
| map_at_1 | 35.423 |
| map_at_10 | 47.198 |
| map_at_100 | 48.899 |
| map_at_1000 | 49.004 |
| map_at_3 | 43.114999999999995 |
| map_at_5 | 45.491 |
| mrr_at_1 | 42.918 |
| mrr_at_10 | 53.299 |
| mrr_at_100 | 54.032000000000004 |
| mrr_at_1000 | 54.055 |
| mrr_at_3 | 50.453 |
| mrr_at_5 | 52.205999999999996 |
| ndcg_at_1 | 42.918 |
| ndcg_at_10 | 53.98 |
| ndcg_at_100 | 59.57 |
| ndcg_at_1000 | 60.879000000000005 |
| ndcg_at_3 | 48.224000000000004 |
| ndcg_at_5 | 50.998 |
| precision_at_1 | 42.918 |
| precision_at_10 | 10.299999999999999 |
| precision_at_100 | 1.687 |
| precision_at_1000 | 0.211 |
| precision_at_3 | 22.842000000000002 |
| precision_at_5 | 16.681 |
| recall_at_1 | 35.423 |
| recall_at_10 | 66.824 |
| recall_at_100 | 89.564 |
| recall_at_1000 | 97.501 |
| recall_at_3 | 50.365 |
| recall_at_5 | 57.921 |
MTEB CQADupstackEnglishRetrieval
- Task Type: Retrieval
- Dataset Revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
- Split: test
| Metric Type | Value |
|-------------|-------|
| map_at_1 | 33.205 |
| map_at_10 | 44.859 |
| map_at_100 | 46.135 |
| map_at_1000 | 46.259 |
| map_at_3 | 41.839 |
| map_at_5 | 43.662 |
| mrr_at_1 | 41.146 |
| mrr_at_10 | 50.621 |
| mrr_at_100 | 51.207 |
| mrr_at_1000 | 51.246 |
| mrr_at_3 | 48.535000000000004 |
| mrr_at_5 | 49.818 |
| ndcg_at_1 | 41.146 |
| ndcg_at_10 | 50.683 |
| ndcg_at_100 | 54.82 |
| ndcg_at_1000 | 56.69 |
| ndcg_at_3 | 46.611000000000004 |
| ndcg_at_5 | 48.66 |
| precision_at_1 | 41.146 |
| precision_at_10 | 9.439 |
| precision_at_100 | 1.465 |
| precision_at_1000 | 0.194 |
| precision_at_3 | 22.59 |
| precision_at_5 | 15.86 |
| recall_at_1 | 33.205 |
| recall_at_10 | 61.028999999999996 |
| recall_at_100 | 78.152 |
| recall_at_1000 | 89.59700000000001 |
| recall_at_3 | 49.05 |
| recall_at_5 | 54.836 |
MTEB CQADupstackGamingRetrieval
- Task Type: Retrieval
- Dataset Revision: 4885aa143210c98657558c04aaf3dc47cfb54340
- Split: test
| Metric Type | Value |
|-------------|-------|
| map_at_1 | 41.637 |
| map_at_10 | 55.162 |
| map_at_100 | 56.142 |
| map_at_1000 | 56.188 |
| map_at_3 | 51.564 |
| map_at_5 | 53.696 |
| mrr_at_1 | 47.524 |
| mrr_at_10 | 58.243 |
| mrr_at_100 | 58.879999999999995 |
| mrr_at_1000 | 58.9 |
| mrr_at_3 | 55.69499999999999 |
| mrr_at_5 | 57.284 |
| ndcg_at_1 | 47.524 |
| ndcg_at_10 | 61.305 |
| ndcg_at_100 | 65.077 |
| ndcg_at_1000 | 65.941 |
| ndcg_at_3 | 55.422000000000004 |
| ndcg_at_5 | 58.516 |
| precision_at_1 | 47.524 |
| precision_at_10 | 9.918000000000001 |
| precision_at_100 | 1.276 |
| precision_at_1000 | 0.13899999999999998 |
| precision_at_3 | 24.765 |
| precision_at_5 | 17.204 |
| recall_at_1 | 41.637 |
| recall_at_10 | 76.185 |
| recall_at_100 | 92.149 |
| recall_at_1000 | 98.199 |
| recall_at_3 | 60.856 |
| recall_at_5 | 68.25099999999999 |
MTEB CQADupstackGisRetrieval
- Task Type: Retrieval
- Dataset Revision: 5003b3064772da1887988e05400cf3806fe491f2
- Split: test
| Metric Type | Value |
|-------------|-------|
| map_at_1 | 26.27 |
| map_at_10 | 37.463 |
| map_at_100 | 38.434000000000005 |
| map_at_1000 | 38.509 |
| map_at_3 | 34.226 |
| map_at_5 | 36.161 |
| mrr_at_1 | 28.588 |
| mrr_at_10 | 39.383 |
| mrr_at_100 | 40.23 |
| mrr_at_1000 | 40.281 |
| mrr_at_3 | 36.422 |
| mrr_at_5 | 38.252 |
| ndcg_at_1 | 28.588 |
| ndcg_at_10 | 43.511 |
| ndcg_at_100 | 48.274 |
| ndcg_at_1000 | 49.975 |
| ndcg_at_3 | 37.319 |
| ndcg_at_5 | 40.568 |
| precision_at_1 | 28.588 |
| precision_at_10 | 6.893000000000001 |
| precision_at_100 | 0.9900000000000001 |
| precision_at_1000 | 0.117 |
| precision_at_3 | 16.347 |
| precision_at_5 | 11.661000000000001 |
| recall_at_1 | 26.27 |
| recall_at_10 | 60.284000000000006 |
| recall_at_100 | 81.902 |
| recall_at_1000 | 94.43 |
| recall_at_3 | 43.537 |
| recall_at_5 | 51.475 |
MTEB CQADupstackMathematicaRetrieval
- Task Type: Retrieval
- Dataset Revision: 90fceea13679c63fe563ded68f3b6f06e50061de
- Split: test
| Metric Type | Value |
|-------------|-------|
| map_at_1 | 18.168 |
| map_at_10 | 28.410000000000004 |
| map_at_100 | 29.78 |
| map_at_1000 | 29.892999999999997 |
| map_at_3 | 25.238 |
| map_at_5 | 26.96 |
| mrr_at_1 | 23.507 |
| mrr_at_10 | 33.382 |
| mrr_at_100 | 34.404 |
| mrr_at_1000 | 34.467999999999996 |
| mrr_at_3 | 30.637999999999998 |
| mrr_at_5 | 32.199 |
| ndcg_at_1 | 23.507 |
| ndcg_at_10 | 34.571000000000005 |
| ndcg_at_100 | 40.663 |
| ndcg_at_1000 | 43.236000000000004 |
| ndcg_at_3 | 29.053 |
| ndcg_at_5 | 31.563999999999997 |
| precision_at_1 | 23.507 |
| precision_at_10 | 6.654 |
| precision_at_100 | 1.113 |
| precision_at_1000 | 0.146 |
| precision_at_3 | 14.427999999999999 |
| precision_at_5 | 10.498000000000001 |
📄 License
This model is licensed under the apache - 2.0
license.