SGPT-2.7B Open-Source Sentence Transformer Model - Free Deployment to Assist Sentence Similarity Tasks

SGPT 2.7B Weightedmean Msmarco Specb Bitfit

Developed by Muennighoff

SGPT-2.7B is a sentence transformer model based on the weighted mean method, focusing on sentence similarity tasks, trained on the MSMARCO dataset with BitFit technology applied.

Text Embedding

PyTorch

Downloads 85

Release Time : 3/2/2022

Model Overview

This model is primarily used for sentence similarity calculation and feature extraction tasks, performing well on the MTEB benchmark, suitable for various text classification and retrieval scenarios.

Model Features

Weighted Mean Method

Uses weighted mean techniques to process sentence representations, improving similarity calculation accuracy.

BitFit Technology

Applies BitFit parameter-efficient fine-tuning method, reducing computational resource requirements while maintaining performance.

Multi-task Adaptability

Performs well on various tasks in the MTEB benchmark, including classification, clustering, and retrieval.

Model Capabilities

Sentence similarity calculation

Text feature extraction

Text classification

Information retrieval

Text clustering

Use Cases

E-commerce

Product review classification

Sentiment polarity classification for Amazon product reviews

Achieved 71.44% accuracy on the MTEB Amazon polarity classification task

Counterfactual review detection

Identifying counterfactual reviews on Amazon platform

Achieved 67.57% accuracy on the MTEB Amazon counterfactual classification task

Finance

Bank customer service classification

Classifying bank customer service inquiries

Achieved 83.22% accuracy on the MTEB Banking77 classification task

Academic

Paper clustering

Topic clustering for Arxiv academic papers

Achieved V-measure of 44.72 on the MTEB Arxiv clustering P2P task

🚀 SGPT-2.7B-weightedmean-msmarco-specb-bitfit

This model is designed for sentence similarity tasks, leveraging sentence-transformers for feature extraction and performing well on various MTEB datasets.

📚 Documentation

Model Information

Property	Details
Model Type	SGPT-2.7B-weightedmean-msmarco-specb-bitfit
Pipeline Tag	sentence-similarity
Tags	sentence-transformers, feature-extraction, sentence-similarity, mteb

Results on Datasets

Classification Tasks

Dataset	Accuracy	AP	F1
MTEB AmazonCounterfactualClassification (en)	67.56716417910448	30.75574629595259	61.805121301858655
MTEB AmazonPolarityClassification	71.439575	65.91341330532453	70.90561852619555
MTEB AmazonReviewsClassification (en)	35.748000000000005	-	35.48576287186347
MTEB Banking77Classification	83.21753246753246	-	83.15394543120915

Retrieval Tasks

Dataset	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000
MTEB ArguAna	25.96	41.619	42.673	42.684	26.316	41.772	42.82	42.83	25.96	50.491	54.864999999999995	55.10699999999999	25.96	7.8950000000000005	0.9780000000000001	0.1	25.96	78.947	97.795	99.644
MTEB CQADupstackAndroidRetrieval	30.808999999999997	40.617	41.894999999999996	42.025	37.482	46.497	47.144000000000005	47.189	37.482	46.688	51.726000000000006	53.825	37.482	8.827	1.393	0.186	30.808999999999997	58.47	80.51899999999999	93.809
MTEB CQADupstackEnglishRetrieval	26.962000000000003	36.93	38.102000000000004	38.22	33.567	42.269	42.99	43.033	33.567	42.405	46.847	48.951	33.567	8.032	1.295	0.17600000000000002	26.962000000000003	52.489	71.635	85.141
MTEB CQADupstackGamingRetrieval	36.318	47.97	49.003	49.065999999999995	41.504999999999995	51.431000000000004	52.129000000000005	52.161	41.504999999999995	53.676	57.867000000000004	59.166	41.504999999999995	8.608	1.1560000000000001	0.133	36.318	67.066	85.34	94.491
MTEB CQADupstackGisRetrieval	22.167	29.543999999999997	30.579	30.669999999999998	24.068	31.237	32.222	32.292	24.068	33.973	39.135	41.443999999999996	24.068	5.299	0.823	0.106	22.167	46.115	69.867	87.234
MTEB CQADupstackMathematicaRetrieval	12.033000000000001	19.314	20.562	20.695	14.801	22.74	23.876	23.949	14.801	24.038	30.186	33.321	14.801	4.776	-	-	12.033000000000001	30.75574629595259	61.805121301858655	-

Clustering Tasks

Dataset	V-Measure
MTEB ArxivClusteringP2P	44.72125714642202
MTEB ArxivClusteringS2S	35.081451519142064
MTEB BiorxivClusteringP2P	34.41414219680629
MTEB BiorxivClusteringS2S	30.533275862270028

Reranking Task

Dataset	MAP	MRR
MTEB AskUbuntuDupQuestions	59.634661990392054	73.6813525040672

STS Task

Dataset	Cosine Similarity Pearson	Cosine Similarity Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
MTEB BIOSSES	87.42754550496836	84.84289705838664	85.59331970450859	85.8525586184271	85.41233134466698	85.52303303767404

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご