MiniCPM-Embedding-Light Open-Source Model - Free to Use, Supports Chinese and English Semantic Similarity and Classification Tasks

Minicpm Embedding Light

Developed by openbmb

This model has been tested on multiple Chinese and English datasets, including semantic text similarity and classification tasks.

Large Language Model

Transformers

#Multilingual text similarity #E-commerce review classification #Low-precision semantic matching

Downloads 2,172

Release Time : 1/17/2025

Model Overview

This model is primarily used for semantic text similarity and classification tasks, supporting both Chinese and English.

Model Features

Multilingual support

Supports semantic text similarity and classification tasks in both Chinese and English.

Multi-task capability

Capable of handling various tasks such as semantic text similarity and classification.

Model Capabilities

Semantic text similarity calculation

Text classification

Multilingual processing

Use Cases

E-commerce

Amazon review classification

Performs sentiment classification on Amazon product reviews.

Classification accuracy for English reviews is 47.95%, and for Chinese reviews is 43.92%.

Semantic analysis

Semantic text similarity calculation

Calculates the semantic similarity between two pieces of text.

Cosine Pearson value is 31.60 on the AFQMC dataset and 40.90 on the ATEC dataset.

🚀 Model Evaluation Results

This document presents the evaluation results of a model on various datasets across multiple tasks. The model's performance is measured using a range of metrics, providing a comprehensive view of its capabilities.

📚 Documentation

Model Information

Model Name: no_model_name_available

Evaluation Results

1. Semantic Textual Similarity (STS) Tasks

Dataset	Split	Main Score	Cosine Pearson	Cosine Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman	Pearson	Spearman
MTEB AFQMC (default)	validation	32.26566089398552	31.60219205269865	32.26566089398552	31.38659295608159	32.265680997074284	31.012318343485934	31.881347232593882	31.60219205269865	32.26566089398552
MTEB ATEC (default) - test	test	40.342262626966686	40.89963324512739	40.342262626966686	43.26579075620696	40.34226375259283	43.09428997760782	40.132604575720485	40.89963324512739	40.342262626966686
MTEB ATEC (default) - validation	validation	41.15073385665892	40.97674579633659	41.15073385665892	43.12674145119401	41.15073497290901	43.016332350517416	40.99128368771293	40.97674579633659	41.15073385665892
MTEB BIOSSES (default)	test	88.4670760233051	90.16604991710759	88.4670760233051	89.02378164860428	88.4670760233051	88.8866912507422	88.2755053931781	90.16604991710759	88.4670760233051
MTEB BQ (default) - test	test	49.77342815602789	49.45233904713201	49.77342815602789	49.13579036236359	49.77342122767529	49.01322677955527	49.702538779772226	49.45233904713201	49.77342815602789
MTEB BQ (default) - validation	validation	54.24325202324013	53.43473222697715	54.24325202324013	53.4053341221681	54.2432485591385	53.31602762068146	54.180811590825925	53.43473222697715	54.24325202324013

2. Classification Tasks

Dataset	Split	Main Score	Accuracy	F1	F1 Weighted
MTEB AmazonCounterfactualClassification (en)	test	74.1492537313433	74.1492537313433	67.93641050300623	76.47946936836382
MTEB AmazonPolarityClassification (default)	test	92.57937499999998	92.57937499999998	92.57389464257594	92.57389464257594
MTEB AmazonReviewsClassification (en) - test	test	47.95399999999999	47.95399999999999	45.23480325168402	45.23480325168403
MTEB AmazonReviewsClassification (zh) - test	test	43.916000000000004	43.916000000000004	40.79038102586015	40.79038102586015
MTEB AmazonReviewsClassification (en) - validation	validation	47.716	47.716	44.97469896514136	44.97469896514136
MTEB AmazonReviewsClassification (zh) - validation	validation	43.016000000000005	43.016000000000005	39.88062282479835	39.88062282479835
MTEB Banking77Classification (default)	test	82.11038961038962	82.11038961038962	81.50275371635729	81.50275371635732

3. Retrieval Task

Dataset	Split	Main Score	MAP@1	MAP@10	MAP@100	MAP@1000	MAP@20	MAP@3	MAP@5	MRR@1	MRR@10	MRR@100	MRR@1000	MRR@20	MRR@3	MRR@5
MTEB ArguAna (default)	test	55.97299999999999	31.009999999999998	46.951	47.788000000000004	47.794	47.656	41.69	44.795	31.57894736842105	47.150336426652245	48.00003421265431	48.006517491673485	47.86823495425013	41.90374585111427	45.00474158368897

4. Clustering Tasks

Dataset	Split	Main Score	V-Measure	V-Measure Std
MTEB ArxivClusteringP2P (default)	test	48.31683216128774	48.31683216128774	13.795207109799703
MTEB ArxivClusteringS2S (default)	test	40.2951016935384	40.2951016935384	14.193710444297869
MTEB BiorxivClusteringP2P (default)	test	39.85718105201471	39.85718105201471	0.9098592525717781
MTEB BiorxivClusteringS2S (default)	test	34.901371726743854	34.901371726743854	0.49131958662099773
MTEB CLSClusteringP2P (default)	test	42.580911514601844	42.580911514601844	1.3262494874619402
MTEB CLSClusteringS2S (default)	test	38.36369670561906	38.36369670561906	1.3030031287521193

5. Reranking Task

Dataset	Split	Main Score	MAP	MRR
MTEB AskUbuntuDupQuestions (default)	test	60.45095169935259	60.45095169935259	73.43567251461988

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご