SGPT-125M Open Source Sentence Transformer Model - Free Deployment, Accurately Complete Sentence Similarity Tasks

SGPT 125M Weightedmean Msmarco Specb Bitfit

Developed by Muennighoff

SGPT-125M is a sentence transformer model optimized with weighted mean and bitfit techniques, focusing on sentence similarity tasks.

Text Embedding

PyTorch

#Multilingual sentence similarity #Text classification optimization #Low-resource efficiency

Downloads 4,086

Release Time : 3/2/2022

Model Overview

This model is primarily used for sentence similarity calculation and feature extraction, supporting multilingual text processing tasks.

Model Features

Multilingual support

Supports processing multiple languages including English, German, Spanish, French, Japanese, and Chinese.

Weighted mean technique

Uses weighted mean method to optimize sentence representation and improve similarity calculation performance.

Bitfit optimization

Employs bitfit technology for model fine-tuning to enhance performance on specific tasks.

Model Capabilities

Sentence similarity calculation

Text feature extraction

Multilingual text processing

Classification tasks

Clustering tasks

Retrieval tasks

Use Cases

E-commerce

Product review classification

Classify product reviews on platforms like Amazon.

Achieved 31.17% accuracy in English for MTEB Amazon review classification task

Counterfactual classification

Identify counterfactual statements in Amazon product descriptions.

Achieved 61.24% accuracy in English for MTEB Amazon counterfactual classification task

Academic research

Paper clustering

Cluster academic papers from arXiv and biorxiv.

Achieved V-measure of 39.71 in MTEB Arxiv clustering P2P task

Q&A systems

Duplicate question identification

Identify duplicate questions on AskUbuntu forums.

Achieved average precision of 55.84% in MTEB AskUbuntu duplicate questions task

🚀 SGPT-125M-weightedmean-msmarco-specb-bitfit

This model is designed for sentence similarity tasks, leveraging sentence-transformers for feature extraction. It has been evaluated on multiple datasets across various tasks such as Classification, Retrieval, Clustering, Reranking, and STS, with detailed performance metrics provided.

📚 Documentation

Model Information

Property	Details
Pipeline Tag	sentence-similarity
Tags	sentence-transformers, feature-extraction, sentence-similarity, mteb
Model Name	SGPT-125M-weightedmean-msmarco-specb-bitfit

Evaluation Results

Classification Tasks

Task	Dataset	Language	Accuracy	AP	F1
Classification	MTEB AmazonCounterfactualClassification	en	61.23880597014926	25.854431650388644	55.751862762818604
Classification	MTEB AmazonCounterfactualClassification	de	56.88436830835117	72.67279104379772	54.449840243786404
Classification	MTEB AmazonCounterfactualClassification	en-ext	58.27586206896551	14.067357642500387	48.172318518691334
Classification	MTEB AmazonCounterfactualClassification	ja	54.64668094218415	11.776694555054965	44.526622834078765
Classification	MTEB AmazonPolarityClassification	default	65.401225	60.22809958678552	65.0251824898292
Classification	MTEB AmazonReviewsClassification	en	31.165999999999993	-	30.908870050167437
Classification	MTEB AmazonReviewsClassification	de	24.79	-	24.5833598854121
Classification	MTEB AmazonReviewsClassification	es	26.643999999999995	-	26.39012792213563
Classification	MTEB AmazonReviewsClassification	fr	26.386000000000003	-	26.276867791454873
Classification	MTEB AmazonReviewsClassification	ja	22.078000000000003	-	21.797960290226843
Classification	MTEB AmazonReviewsClassification	zh	24.274	-	23.887054434822627
Classification	MTEB Banking77Classification	default	77.70454545454545	-	77.6929000113803

Retrieval Tasks

Task	Dataset	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000
Retrieval	MTEB ArguAna	22.404	36.845	37.945	37.966	22.902	37.034	38.134	38.155	22.404	45.425	50.354	50.873999999999995	22.404	7.303999999999999	0.951	0.099	22.404	73.044	95.092	99.075
Retrieval	MTEB CQADupstackAndroidRetrieval	22.139	28.839	30.023	30.153000000000002	26.466	33.495000000000005	34.416999999999994	34.485	26.466	33.372	38.7	41.696	26.466	6.037	1.0670000000000002	0.16199999999999998	22.139	42.39	65.427	86.04899999999999
Retrieval	MTEB CQADupstackEnglishRetrieval	20.652	27.558	28.473	28.577	25.223000000000003	31.966	32.664	32.724	25.223000000000003	31.694	35.662	38.092	25.223000000000003	5.777	0.9730000000000001	0.13999999999999999	20.652	39.367999999999995	56.485	73.292
Retrieval	MTEB CQADupstackGamingRetrieval	25.180000000000003	34.579	35.589999999999996	35.68	29.467	37.967	38.800000000000004	38.858	29.467	39.796	44.531	46.666000000000004	29.467	6.601999999999999	0.9900000000000001	0.124	25.180000000000003	52.269	73.574	89.141

Clustering Tasks

Task	Dataset	V-Measure
Clustering	MTEB ArxivClusteringP2P	39.70858340673288
Clustering	MTEB ArxivClusteringS2S	28.242847713721048
Clustering	MTEB BiorxivClusteringP2P	33.63260395543984
Clustering	MTEB BiorxivClusteringS2S	27.038042665369925

Reranking Task

Task	Dataset	MAP	MRR
Reranking	MTEB AskUbuntuDupQuestions	55.83700395192393	70.3891307215407

STS Task

Task	Dataset	Cosine Similarity Pearson	Cosine Similarity Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
STS	MTEB BIOSSES	79.25366801756223	75.20954502580506	78.79900722991617	77.79996549607588	78.18408109480399	76.85958262303106

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご