Open-source model st-polish-kartonberta-base-alpha-v1 - Freely implement Polish sentence similarity calculation and feature extraction

St Polish Kartonberta Base Alpha V1

Developed by OrlikB

This is a Polish sentence transformer model based on the KartonBERTa architecture, primarily used for sentence similarity calculation and feature extraction tasks.

Text Embedding

Transformers

Other#Polish sentence similarity #Text feature extraction #Multi-task evaluation

Downloads 3,494

Release Time : 11/12/2023

Model Overview

The model specializes in Polish text processing, capable of converting sentences into high-dimensional vector representations for tasks such as sentence similarity calculation, clustering, and classification.

Model Features

Polish language optimization

Specially optimized for Polish text, providing better language understanding capabilities

Sentence similarity calculation

Accurately calculates semantic similarity between Polish sentences

Multi-task support

Supports various natural language processing tasks such as clustering, classification, and retrieval

Model Capabilities

Sentence feature extraction

Sentence similarity calculation

Text clustering

Text classification

Information retrieval

Use Cases

Text analysis

Review classification

Sentiment classification of Polish product reviews

Achieved 40.19% accuracy on the AllegroReviews dataset

Semantic similarity judgment

Determining whether two Polish sentences express the same meaning

Achieved 89.2% accuracy on the CDSC-E dataset

Information retrieval

Document retrieval

Retrieving relevant Polish documents based on queries

Achieved 54.54% MRR@100 on the DBPedia-PL dataset

🚀 st-polish-kartonberta-base-alpha-v1

This is a model for sentence similarity tasks, with performance metrics on multiple datasets.

📚 Documentation

Model Information

Property	Details
Model Type	st-polish-kartonberta-base-alpha-v1
Task	Sentence Similarity, Clustering, Classification, Retrieval, PairClassification, STS
Tags	sentence-transformers, feature-extraction, sentence-similarity, transformers, mteb
License	LGPL
Language	Polish (pl)

Model Performance

1. Clustering Task

Dataset: PL-MTEB/8tags-clustering (MTEB 8TagsClustering, test split)
Metric: v_measure = 32.85180358455615

2. Classification Tasks

Dataset 1: PL-MTEB/allegro-reviews (MTEB AllegroReviews, test split)
- Metrics:
  - accuracy = 40.188866799204774
  - f1 = 34.71127012684797
Dataset 2: PL-MTEB/cbd (MTEB CBD, test split)
- Metrics:
  - accuracy = 67.69000000000001
  - ap = 21.078799692467182
  - f1 = 56.80107173953953
Dataset 3: mteb/amazon_massive_intent (MTEB MassiveIntentClassification (pl), test split)
- Metrics:
  - accuracy = 68.19771351714861
  - f1 = 64.75039989217822
Dataset 4: mteb/amazon_massive_scenario (MTEB MassiveScenarioClassification (pl), test split)
- Metrics:
  - accuracy = 73.9677202420982
  - f1 = 73.72287107577753

3. Retrieval Tasks

Dataset 1: arguana-pl (MTEB ArguAna-PL, test split)
- Metrics:
  - map_at_1 = 30.939
  - map_at_10 = 47.467999999999996
  - map_at_100 = 48.303000000000004
  - map_at_1000 = 48.308
  - map_at_3 = 43.22
  - map_at_5 = 45.616
  - mrr_at_1 = 31.863000000000003
  - mrr_at_10 = 47.829
  - mrr_at_100 = 48.664
  - mrr_at_1000 = 48.67
  - mrr_at_3 = 43.492
  - mrr_at_5 = 46.006
  - ndcg_at_1 = 30.939
  - ndcg_at_10 = 56.058
  - ndcg_at_100 = 59.562000000000005
  - ndcg_at_1000 = 59.69799999999999
  - ndcg_at_3 = 47.260000000000005
  - ndcg_at_5 = 51.587
  - precision_at_1 = 30.939
  - precision_at_10 = 8.329
  - precision_at_100 = 0.984
  - precision_at_1000 = 0.1
  - precision_at_3 = 19.654
  - precision_at_5 = 13.898
  - recall_at_1 = 30.939
  - recall_at_10 = 83.286
  - recall_at_100 = 98.43499999999999
  - recall_at_1000 = 99.502
  - recall_at_3 = 58.962
  - recall_at_5 = 69.488
Dataset 2: dbpedia-pl (MTEB DBPedia-PL, test split)
- Metrics:
  - map_at_1 = 5.8709999999999996
  - map_at_10 = 12.486
  - map_at_100 = 16.897000000000002
  - map_at_1000 = 18.056
  - map_at_3 = 8.958
  - map_at_5 = 10.57
  - mrr_at_1 = 44.0
  - mrr_at_10 = 53.830999999999996
  - mrr_at_100 = 54.54
  - mrr_at_1000 = 54.568000000000005
  - mrr_at_3 = 51.87500000000001
  - mrr_at_5 = 53.113
  - ndcg_at_1 = 34.625
  - ndcg_at_10 = 26.996
  - ndcg_at_100 = 31.052999999999997
  - ndcg_at_1000 = 38.208
  - ndcg_at_3 = 29.471000000000004
  - ndcg_at_5 = 28.364
  - precision_at_1 = 44.0
  - precision_at_10 = 21.45
  - precision_at_100 = 6.837
  - precision_at_1000 = 1.6019999999999999
  - precision_at_3 = 32.333
  - precision_at_5 = 27.800000000000004
  - recall_at_1 = 5.8709999999999996
  - recall_at_10 = 17.318
  - recall_at_100 = 36.854
  - recall_at_1000 = 60.468999999999994
  - recall_at_3 = 10.213999999999999
  - recall_at_5 = 13.364
Dataset 3: fiqa-pl (MTEB FiQA-PL, test split)
- Metrics:
  - map_at_1 = 10.289
  - map_at_10 = 18.285999999999998
  - map_at_100 = 19.743
  - map_at_1000 = 19.964000000000002
  - map_at_3 = 15.193000000000001
  - map_at_5 = 16.962
  - mrr_at_1 = 21.914
  - mrr_at_10 = 30.653999999999996
  - mrr_at_100 = 31.623
  - mrr_at_1000 = 31.701
  - mrr_at_3 = 27.855
  - mrr_at_5 = 29.514000000000003
  - ndcg_at_1 = 21.914
  - ndcg_at_10 = 24.733
  - ndcg_at_100 = 31.253999999999998
  - ndcg_at_1000 = 35.617
  - ndcg_at_3 = 20.962
  - ndcg_at_5 = 22.553
  - precision_at_1 = 21.914
  - precision_at_10 = 7.346
  - precision_at_100 = 1.389
  - precision_at_1000 = 0.214
  - precision_at_3 = 14.352
  - precision_at_5 = 11.42
  - recall_at_1 = 10.289
  - recall_at_10 = 31.459
  - recall_at_100 = 56.854000000000006
  - recall_at_1000 = 83.722
  - recall_at_3 = 19.457
  - recall_at_5 = 24.767
Dataset 4: hotpotqa-pl (MTEB HotpotQA-PL, test split)
- Metrics:
  - map_at_1 = 29.669
  - map_at_10 = 41.615
  - map_at_100 = 42.571999999999996
  - map_at_1000 = 42.662
  - map_at_3 = 38.938
  - map_at_5 = 40.541
  - mrr_at_1 = 59.338
  - mrr_at_10 = 66.93900000000001
  - mrr_at_100 = 67.361
  - mrr_at_1000 = 67.38499999999999
  - mrr_at_3 = 65.384
  - mrr_at_5 = 66.345
  - ndcg_at_1 = 59.338
  - ndcg_at_10 = 50.607
  - ndcg_at_100 = 54.342999999999996
  - ndcg_at_1000 = 56.286
  - ndcg_at_3 = 46.289
  - ndcg_at_5 = 48.581
  - precision_at_1 = 59.338
  - precision_at_10 = 10.585
  - precision_at_100 = 1.353
  - precision_at_1000 = 0.161
  - precision_at_3 = 28.877000000000002
  - precision_at_5 = 19.133
  - recall_at_1 = 29.669
  - recall_at_10 = 52.92400000000001
  - recall_at_100 = 67.657
  - recall_at_1000 = 80.628
  - recall_at_3 = 43.315
  - recall_at_5 = 47.833
Dataset 5: msmarco-pl (MTEB MSMARCO-PL, test split)
- Metrics:
  - map_at_1 = 0.997
  - map_at_10 = 7.481999999999999
  - map_at_100 = 20.208000000000002
  - map_at_1000 = 25.601000000000003
  - map_at_3 = 3.055
  - map_at_5 = 4.853
  - mrr_at_1 = 55.814
  - mrr_at_10 = 64.651
  - mrr_at_100 = 65.003
  - mrr_at_1000 = 65.05199999999999
  - mrr_at_3 = 62.403
  - mrr_at_5 = 64.031
  - ndcg_at_1 = 44.186
  - ndcg_at_10 = 43.25
  - ndcg_at_100 = 40.515
  - ndcg_at_1000 = 48.345
  - ndcg_at_3 = 45.829
  - ndcg_at_5 = 46.477000000000004
  - precision_at_1 = 55.814
  - precision_at_10 = 50.465
  - precision_at_100 = 25.419000000000004
  - precision_at_1000 = 5.0840000000000005
  - precision_at_3 = 58.14
  - precision_at_5 = 57.67400000000001
  - recall_at_1 = 0.997
  - recall_at_10 = 8.985999999999999
  - recall_at_100 = 33.221000000000004
  - recall_at_1000 = 58.836999999999996
  - recall_at_3 = 3.472
  - recall_at_5 = 5.545
Dataset 6: nfcorpus-pl (MTEB NFCorpus-PL, test split)
- Metrics:
  - map_at_1 = 5.167
  - map_at_10 = 10.791
  - map_at_100 = 14.072999999999999
  - map_at_1000 = 15.568000000000001
  - map_at_3 = 7.847999999999999
  - map_at_5 = 9.112
  - mrr_at_1 = 42.105
  - mrr_at_10 = 49.933
  - mrr_at_100 = 50.659
  - mrr_at_1000 = 50.705
  - mrr_at_3 = 47.988
  - mrr_at_5 = 49.056
  - ndcg_at_1 = 39.938
  - ndcg_at_10 = 31.147000000000002
  - ndcg_at_100 = 29.336000000000002
  - ndcg_at_1000 = 38.147
  - ndcg_at_3 = 35.607
  - ndcg_at_5 = 33.725
  - precision_at_1 = 41.486000000000004
  - precision_at_10 = 23.901
  - precision_at_100 = 7.960000000000001
  - precision_at_1000 = 2.086
  - precision_at_3 = 33.437
  - precision_at_5 = 29.598000000000003
  - recall_at_1 = 5.167
  - recall_at_10 = 14.872
  - recall_at_100 = 28.146
  - recall_at_1000 = 48.472
  - recall_at_3 = 7.057
  - recall_at_5 = 9.496

4. PairClassification Task

Dataset: PL-MTEB/cdsce-pairclassification (MTEB CDSC-E, test split)
- Metrics:
  - cos_sim_accuracy = 89.2
  - cos_sim_ap = 79.11674608786898
  - cos_sim_f1 = 68.83468834688347
  - cos_sim_precision = 70.94972067039106
  - cos_sim_recall = 66.84210526315789
  - dot_accuracy = 89.2
  - dot_ap = 79.11674608786898
  - dot_f1 = 68.83468834688347
  - dot_precision = 70.94972067039106
  - dot_recall = 66.84210526315789
  - euclidean_accuracy = 89.2
  - euclidean_ap = 79.11674608786898
  - euclidean_f1 = 68.83468834688347
  - euclidean_precision = 70.94972067039106
  - euclidean_recall = 66.84210526315789
  - manhattan_accuracy = 89.1
  - manhattan_ap = 79.1220443374692
  - manhattan_f1 = 69.02173913043478
  - manhattan_precision = 71.34831460674157
  - manhattan_recall = 66.84210526315789
  - max_accuracy = 89.2
  - max_ap = 79.1220443374692
  - max_f1 = 69.02173913043478

5. STS Task

Dataset: PL-MTEB/cdscr-sts (MTEB CDSC-R, test split)
- Metrics:
  - cos_sim_pearson = 91.41534744278998
  - cos_sim_spearman = 92.12681551821147
  - euclidean_pearson = 91.74369794485992
  - euclidean_spearman = 92.12685848456046
  - manhattan_pearson = 91.66651938751657
  - manhattan_spearman = 92.057603126734

📄 License

This project is licensed under the LGPL license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご