mmlw-e5-base Open-source Feature Extraction Model - Free Deployment Support for Polish Sentence Similarity Tasks

Mmlw E5 Base

Developed by sdadas

mmlw-e5-base is a feature extraction model based on sentence transformers, focusing on sentence similarity tasks and supporting Polish.

Text Embedding

Transformers

OtherOpen Source License:Apache-2.0 #Polish sentence similarity #Multi-task evaluation benchmark #High-precision semantic retrieval

Downloads 37

Release Time : 11/17/2023

Model Overview

This model is primarily used for sentence similarity calculation and feature extraction, suitable for various natural language processing tasks such as text clustering, classification, and retrieval.

Model Features

Multi-task support

The model supports multiple tasks, including clustering, classification, retrieval, and sentence similarity calculation.

Polish language optimization

Optimized for Polish, demonstrating excellent performance on multiple Polish datasets.

High-performance feature extraction

Capable of efficiently extracting sentence-level feature representations for downstream tasks.

Model Capabilities

Sentence similarity calculation

Text feature extraction

Text clustering

Text classification

Information retrieval

Use Cases

Text analysis

Product review classification

Sentiment classification for Polish product reviews

Achieved 36.38% accuracy on the MTEB AllegroReviews dataset

Legal clause classification

Identifying abusive content in legal clauses

Achieved 64.50% accuracy on the MTEB PAC dataset

Information retrieval

Question answering system

Building a Polish question answering system

Achieved 24.67% map@1 on the MTEB NQ-PL dataset

Document retrieval

Polish document retrieval system

Achieved 8.27% map@1 on the MTEB DBPedia-PL dataset

🚀 mmlw-e5-base

This is a model for sentence similarity tasks. It belongs to the sentence-transformers family and can be used for feature extraction and various NLP tasks like clustering, classification, and retrieval.

✨ Features

Multi - task Capability: Suitable for multiple NLP tasks including Clustering, Classification, Retrieval, PairClassification, and STS.
Performance Metrics: Provides a wide range of performance metrics on different datasets, such as v_measure, accuracy, f1, map_at_n, mrr_at_n, ndcg_at_n, precision_at_n, and recall_at_n.

📚 Documentation

Model Information

Property	Details
Model Type	Sentence - similarity model based on sentence - transformers
Tags	sentence - transformers, feature - extraction, sentence - similarity, transformers, mteb
Pipeline Tag	sentence - similarity

Results on Different Tasks and Datasets

1. Clustering

Dataset: PL - MTEB/8tags - clustering (MTEB 8TagsClustering, test split)
Metric: v_measure = 30.249113010261492

2. Classification

Dataset 1: PL - MTEB/allegro - reviews (MTEB AllegroReviews, test split)
- Metrics: accuracy = 36.3817097415507, f1 = 32.77742158736663
Dataset 2: PL - MTEB/cbd (MTEB CBD, test split)
- Metrics: accuracy = 65.53999999999999, ap = 19.75395945379771, f1 = 55.00481388401326
Dataset 3: mteb/amazon_massive_intent (MTEB MassiveIntentClassification (pl), test split)
- Metrics: accuracy = 70.9280430396772, f1 = 68.7099581466286
Dataset 4: mteb/amazon_massive_scenario (MTEB MassiveScenarioClassification (pl), test split)
- Metrics: accuracy = 74.76126429051783, f1 = 74.72274307018111

3. Retrieval

Dataset 1: arguana - pl (MTEB ArguAna - PL, test split)
- Metrics: map_at_1 = 32.646, map_at_10 = 49.488, map_at_100 = 50.190999999999995, map_at_1000 = 50.194, map_at_3 = 44.749, map_at_5 = 47.571999999999996, mrr_at_1 = 34.211000000000006, mrr_at_10 = 50.112, mrr_at_100 = 50.836000000000006, mrr_at_1000 = 50.839, mrr_at_3 = 45.614, mrr_at_5 = 48.242000000000004, ndcg_at_1 = 32.646, ndcg_at_10 = 58.396, ndcg_at_100 = 61.285000000000004, ndcg_at_1000 = 61.358999999999995, ndcg_at_3 = 48.759, ndcg_at_5 = 53.807, precision_at_1 = 32.646, precision_at_10 = 8.663, precision_at_100 = 0.9900000000000001, precision_at_1000 = 0.1, precision_at_3 = 20.128, precision_at_5 = 14.509, recall_at_1 = 32.646, recall_at_10 = 86.629, recall_at_100 = 99.004, recall_at_1000 = 99.57300000000001, recall_at_3 = 60.38400000000001, recall_at_5 = 72.54599999999999
Dataset 2: dbpedia - pl (MTEB DBPedia - PL, test split)
- Metrics: map_at_1 = 8.268, map_at_10 = 17.391000000000002, map_at_100 = 24.266, map_at_1000 = 25.844, map_at_3 = 12.636, map_at_5 = 14.701, mrr_at_1 = 62.74999999999999, mrr_at_10 = 70.25200000000001, mrr_at_100 = 70.601, mrr_at_1000 = 70.613, mrr_at_3 = 68.083, mrr_at_5 = 69.37100000000001, ndcg_at_1 = 51.87500000000001, ndcg_at_10 = 37.185, ndcg_at_100 = 41.949, ndcg_at_1000 = 49.523, ndcg_at_3 = 41.556, ndcg_at_5 = 39.278, precision_at_1 = 63.24999999999999, precision_at_10 = 29.225, precision_at_100 = 9.745, precision_at_1000 = 2.046, precision_at_3 = 43.833, precision_at_5 = 37.9, recall_at_1 = 8.268, recall_at_10 = 22.542, recall_at_100 = 48.154, recall_at_1000 = 72.62100000000001, recall_at_3 = 13.818, recall_at_5 = 17.137
Dataset 3: fiqa - pl (MTEB FiQA - PL, test split)
- Metrics: map_at_1 = 16.489, map_at_10 = 26.916, map_at_100 = 28.582, map_at_1000 = 28.774, map_at_3 = 23.048, map_at_5 = 24.977, mrr_at_1 = 33.642, mrr_at_10 = 41.987, mrr_at_100 = 42.882, mrr_at_1000 = 42.93, mrr_at_3 = 39.48, mrr_at_5 = 40.923, ndcg_at_1 = 33.488, ndcg_at_10 = 34.528, ndcg_at_100 = 41.085, ndcg_at_1000 = 44.474000000000004, ndcg_at_3 = 30.469, ndcg_at_5 = 31.618000000000002, precision_at_1 = 33.488, precision_at_10 = 9.783999999999999, precision_at_100 = 1.6389999999999998, precision_at_1000 = 0.22699999999999998, precision_at_3 = 20.525, precision_at_5 = 15.093, recall_at_1 = 16.489, recall_at_10 = 42.370000000000005, recall_at_100 = 67.183, recall_at_1000 = 87.211, recall_at_3 = 27.689999999999998, recall_at_5 = 33.408
Dataset 4: hotpotqa - pl (MTEB HotpotQA - PL, test split)
- Metrics: map_at_1 = 37.373, map_at_10 = 57.509, map_at_100 = 58.451, map_at_1000 = 58.524, map_at_3 = 54.064, map_at_5 = 56.257999999999996, mrr_at_1 = 74.895, mrr_at_10 = 81.233, mrr_at_100 = 81.461, mrr_at_1000 = 81.47, mrr_at_3 = 80.124, mrr_at_5 = 80.862, ndcg_at_1 = 74.747, ndcg_at_10 = 66.249, ndcg_at_100 = 69.513, ndcg_at_1000 = 70.896, ndcg_at_3 = 61.312, ndcg_at_5 = 64.132, precision_at_1 = 74.747, precision_at_10 = 13.873, precision_at_100 = 1.641, precision_at_1000 = 0.182, precision_at_3 = 38.987, precision_at_5 = 25.621, recall_at_1 = 37.373, recall_at_10 = 69.365, recall_at_100 = 82.039, recall_at_1000 = 91.148, recall_at_3 = 58.48100000000001, recall_at_5 = 64.051
Dataset 5: msmarco - pl (MTEB MSMARCO - PL, validation split)
- Metrics: map_at_1 = 16.753999999999998, map_at_10 = 26.764, map_at_100 = 27.929, map_at_1000 = 27.994999999999997, map_at_3 = 23.527, map_at_5 = 25.343, mrr_at_1 = 17.192, mrr_at_10 = 27.141, mrr_at_100 = 28.269, mrr_at_1000 = 28.327999999999996, mrr_at_3 = 23.906, mrr_at_5 = 25.759999999999998, ndcg_at_1 = 17.177999999999997, ndcg_at_10 = 32.539, ndcg_at_100 = 38.383, ndcg_at_1000 = 40.132, ndcg_at_3 = 25.884, ndcg_at_5 = 29.15, precision_at_1 = 17.177999999999997, precision_at_10 = 5.268, precision_at_100 = 0.823, precision_at_1000 = 0.097, precision_at_3 = 11.122, precision_at_5 = 8.338, recall_at_1 = 16.753999999999998, recall_at_10 = 50.388, recall_at_100 = 77.86999999999999, recall_at_1000 = 91.55, recall_at_3 = 32.186, recall_at_5 = 40.048
Dataset 6: nfcorpus - pl (MTEB NFCorpus - PL, test split)
- Metrics: map_at_1 = 5.348, map_at_10 = 12.277000000000001, map_at_100 = 15.804000000000002, map_at_1000 = 17.277, map_at_3 = 8.783000000000001, map_at_5 = 10.314, mrr_at_1 = 43.963, mrr_at_10 = 52.459999999999994, mrr_at_100 = 53.233, mrr_at_1000 = 53.26499999999999, mrr_at_3 = 50.464, mrr_at_5 = 51.548, ndcg_at_1 = 40.711999999999996, ndcg_at_10 = 33.709, ndcg_at_100 = 31.398, ndcg_at_1000 = 40.042, ndcg_at_3 = 37.85, ndcg_at_5 = 36.260999999999996, precision_at_1 = 43.344, precision_at_10 = 25.851000000000003, precision_at_100 = 8.279, precision_at_1000 = 2.085, precision_at_3 = 36.326, precision_at_5 = 32.074000000000005, recall_at_1 = 5.348, recall_at_10 = 16.441, recall_at_100 = 32.975, recall_at_1000 =... (incomplete in original, assumed to be continued)

4. PairClassification

Dataset: PL - MTEB/cdsce - pairclassification (MTEB CDSC - E, test split)
- Metrics: cos_sim_accuracy = 89.5, cos_sim_ap = 77.26879308078568, cos_sim_f1 = 65.13157894736842, cos_sim_precision = 86.8421052631579, cos_sim_recall = 52.10526315789473, dot_accuracy = 88.0, dot_ap = 69.17235659054914, dot_f1 = 65.71428571428571, dot_precision = 71.875, dot_recall = 60.526315789473685, euclidean_accuracy = 89.5, euclidean_ap = 77.1905400565015, euclidean_f1 = 64.91803278688525, euclidean_precision = 86.08695652173914, euclidean_recall = 52.10526315789473, manhattan_accuracy = 89.5, manhattan_ap = 77.19531778873724, manhattan_f1 = 64.72491909385113, manhattan_precision = 84.03361344537815, manhattan_recall = 52.63157894736842, max_accuracy = 89.5, max_ap = 77.26879308078568, max_f1 = 65.71428571428571

5. STS

Dataset: PL - MTEB/cdscr - sts (MTEB CDSC - R, test split)
- Metrics: cos_sim_pearson = 93.18498922236566, cos_sim_spearman = 93.26224500108704, euclidean_pearson = 92.25462061070286, euclidean_spearman = 93.18623989769242, manhattan_pearson = 92.16291103586255, manhattan_spearman = 93.14403078934417

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご