e5-base-4k Open-source Natural Language Processing Model - Empowering Diverse Tasks Such as Text Classification and Retrieval

E5 Base 4k

Developed by dwzhu

e5-base-4k is a model that performs excellently in multiple natural language processing tasks, especially in tasks such as text classification, retrieval, clustering, and semantic text similarity.

Text Embedding

Transformers

EnglishOpen Source License:MIT #Multi-task text processing #High-precision semantic similarity #Cross-domain retrieval optimization

Downloads 3,029

Release Time : 3/28/2024

Model Overview

e5-base-4k is a multi-functional natural language processing model suitable for various tasks such as text classification, information retrieval, text clustering, and semantic similarity calculation.

Model Features

Excellent multi-task performance

Performs excellently in various tasks such as text classification, retrieval, clustering, and semantic similarity.

Extensive evaluation

Comprehensively evaluated on multiple standard datasets such as MTEB.

High accuracy

Achieves high accuracy in multiple classification tasks. For example, the classification accuracy of AmazonPolarity reaches 92.81%.

Model Capabilities

Text classification

Information retrieval

Text clustering

Semantic similarity calculation

Support for question-answering systems

Use Cases

E-commerce

Product review classification

Classify the sentiment polarity of Amazon product reviews.

Achieves 92.81% accuracy on the AmazonPolarity dataset.

Counterfactual review detection

Identify counterfactual reviews on the Amazon platform.

Accuracy is 77.78%, and the F1 value is 72.12%.

Finance

Bank customer service classification

Classify bank customer service inquiries.

Achieves 83.53% accuracy on the Banking77 dataset.

Academic research

Paper clustering

Perform topic clustering on arXiv and BioRxiv papers.

The V-measure reaches 46.10% on the arXiv dataset.

Question-answering systems

Technical question retrieval

Retrieve relevant questions on technical question-answering platforms (such as AskUbuntu).

The mean average precision is 58.98%.

🚀 e5-base-4k

This document presents the performance metrics of the e5-base-4k model on various tasks and datasets in the MTEB (Massive Text Embedding Benchmark) framework. It includes tasks such as Classification, Retrieval, Clustering, Reranking, and STS (Semantic Textual Similarity), providing a comprehensive evaluation of the model's capabilities.

📚 Documentation

Model Information

Tags: mteb, sentence-similarity
Model Name: e5-base-4k

Performance Metrics

Classification Tasks

Dataset Name	Accuracy	AP	F1
MTEB AmazonCounterfactualClassification (en)	77.77611940298506	42.052710266606056	72.12040628266567
MTEB AmazonPolarityClassification	92.81012500000001	89.4213700757244	92.8039091197065
MTEB AmazonReviewsClassification (en)	46.711999999999996	-	46.11544975436018
MTEB Banking77Classification	83.52922077922076	-	83.45298679360866

Retrieval Tasks

Dataset Name	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000	Recall@3	Recall@5
MTEB ArguAna	23.186	36.632999999999996	37.842	37.865	23.400000000000002	36.721	37.937	37.96	23.186	44.49	50.065000000000005	50.629999999999995	23.186	6.97	0.951	0.099	23.186	69.70100000000001	95.092	99.431	44.737	55.761
MTEB CQADupstackAndroidRetrieval	32.361000000000004	43.765	45.224	45.35	40.629	50.458000000000006	51.06699999999999	51.12	40.629	50.376	55.065	57.196000000000005	40.629	9.785	1.562	0.2	32.361000000000004	62.214000000000006	81.464	95.905	47.5	53.69500000000001
MTEB CQADupstackEnglishRetrieval	27.971	37.444	38.607	38.737	35.35	43.441	44.147999999999996	44.196000000000005	35.35	42.903999999999996	47.406	49.588	35.35	8.083	1.313	0.18	27.971	52.492000000000004	71.642	85.488	40.1	45.800000000000004
MTEB CQADupstackGamingRetrieval	39.898	51.819	52.886	52.941	45.391999999999996	55.230000000000004	55.887	55.916	45.391999999999996	57.586999999999996	61.745000000000005	62.83800000000001	45.391999999999996	9.21	1.226	0.136	39.898	71.18900000000001	89.082	96.865	56.907	63.397999999999996
MTEB CQADupstackGisRetrieval	22.706	30.818	32.038	32.123000000000005	24.407	32.555	33.692	33.751	24.407	35.624	41.454	43.556	24.407	5.548	0.8869999999999999	0.11100000000000002	22.706	48.772	75.053	90.731	34.421	41.427
MTEB CQADupstackMathematicaRetrieval	13.424	21.09	22.264999999999997	22.402	16.915	25.258000000000003	26.228	26.31	16.915	26.266000000000002	32.08	35.086	16.915	5.1	0.9329999999999999	0.131	13.424	38.179	63.906	84.933	23.878	30.037999999999997
MTEB CQADupstackPhysicsRetrieval	26.154	35.912	37.211	37.327	32.435	41.411	42.297000000000004	42.345	32.435	41.785	47.469	49.685	32.435	7.642	1.244	0.163	26.154	54.111	78.348	92.996	39.189	45.852
MTEB CQADupstackProgrammersRetrieval	26.308999999999997	35.524	36.774	36.891	31.735000000000003	40.391	41.227000000000004	41.288000000000004	31.735000000000003	41.166000000000004	46.702	49.157000000000004	31.735000000000003	7.5569999999999995	1.2109999999999999	0.16	26.308999999999997	53.083000000000006	76.922	93.767	39.262	44.413000000000004
MTEB CQADupstackRetrieval	24.391250000000003	33.280166666666666	34.49566666666667	34.61533333333333	29.105083333333337	37.44766666666666	38.32491666666667	38.385666666666665	29.105083333333337	38.54358333333333	43.833583333333344	46.215333333333334	29.105083333333337	6.823416666666665	1.1270833333333334	0.15208333333333332	24.391250000000003	49.98808333333333	73.31616666666666	89.96291666666667	36.86666666666667	42.54350000000001
MTEB CQADupstackStatsRetrieval	21.995	28.807	29.813000000000002	29.903000000000002	24.847	31.494	32.381	32.446999999999996	24.847	32.818999999999996	37.835	40.226	24.847	5.244999999999999	0.856	0.11299999999999999	21.995	42.479	65.337	83.23700000000001	31.573	36.684
MTEB CQADupstackTexRetrieval	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]	[Incomplete data]

Clustering Tasks

Dataset Name	V-Measure
MTEB ArxivClusteringP2P	46.10312401440185
MTEB ArxivClusteringS2S	39.67275326095384
MTEB BiorxivClusteringP2P	37.495115019668496
MTEB BiorxivClusteringS2S	32.724792944166765

Reranking Task

Dataset Name	MAP	MRR
MTEB AskUbuntuDupQuestions	58.97793816337376	72.76832431957087

STS Task

Dataset Name	Cosine Similarity Pearson	Cosine Similarity Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
MTEB BIOSSES	83.11646947018187	81.40064994975234	82.37355689019232	81.6777646977348	82.61101422716945	81.80427360442245

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご