SGPT-5.8B Open-source Sentence Transformer Model - Free Deployment to Facilitate Sentence Similarity Tasks

SGPT 5.8B Weightedmean Msmarco Specb Bitfit

Developed by Muennighoff

SGPT-5.8B is a sentence transformer model based on the weighted mean method, focusing on sentence similarity tasks, trained on the msmarco dataset and optimized with specb-bitfit technology

Text Embedding

PyTorch

Downloads 164

Release Time : 3/2/2022

Model Overview

This model is primarily used for sentence similarity calculation and feature extraction, demonstrating excellent performance in the MTEB benchmark and supporting various natural language processing tasks

Model Features

Weighted Mean Method

Uses weighted mean technology to integrate sentence representations, enhancing semantic understanding capabilities

msmarco Dataset Training

Trained on the large-scale msmarco dataset to improve model generalization

specb-bitfit Optimization

Applies specb-bitfit technology for model optimization, improving computational efficiency

Multi-task Performance

Demonstrates outstanding performance across multiple tasks in the MTEB benchmark

Model Capabilities

Sentence similarity calculation

Text feature extraction

Semantic retrieval

Text classification

Clustering analysis

Question answering reranking

Use Cases

E-commerce

Product Review Classification

Sentiment analysis and classification of Amazon product reviews

Achieved 39.19% accuracy in the MTEB Amazon review classification task

Counterfactual Analysis

Identifying counterfactual reviews on Amazon platform

Achieved 69.22% accuracy in the MTEB Amazon counterfactual classification task

Academic Research

Paper Clustering

Topic clustering for arXiv and BioRxiv academic papers

Achieved a V-measure of 45.59 in the arXiv P2P clustering task

Technical Support

Duplicate Question Detection

Identifying duplicate technical questions in AskUbuntu forums

Achieved an average precision of 61.63% in reranking tasks

🚀 SGPT-5.8B-weightedmean-msmarco-specb-bitfit

This model is designed for sentence similarity tasks, offering feature extraction capabilities and performing well in various MTEB benchmarks.

📚 Documentation

Model Information

Property	Details
Pipeline Tag	sentence-similarity
Tags	sentence-transformers, feature-extraction, sentence-similarity, mteb
Model Name	SGPT-5.8B-weightedmean-msmarco-specb-bitfit

Benchmark Results

Classification Tasks

Task	Dataset	Accuracy	AP	F1
Classification	MTEB AmazonCounterfactualClassification (en)	69.22388059701493	32.04724673950256	63.25719825770428
Classification	MTEB AmazonPolarityClassification	71.26109999999998	66.16336378255403	70.89719145825303
Classification	MTEB AmazonReviewsClassification (en)	39.19199999999999	-	38.580766731113826
Classification	MTEB Banking77Classification	84.49350649350649	-	84.4249343233736

Retrieval Tasks

Task	Dataset	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000	Recall@3	Recall@5
Retrieval	MTEB ArguAna	27.311999999999998	42.620000000000005	43.707	43.714999999999996	27.667	42.737	43.823	43.830999999999996	27.311999999999998	51.37500000000001	55.778000000000006	55.96600000000001	27.311999999999998	7.945	0.9820000000000001	0.1	27.311999999999998	79.445	98.151	99.57300000000001	51.13799999999999	63.727000000000004
Retrieval	MTEB CQADupstackAndroidRetrieval	30.499	41.208	42.638	42.754	37.339	47.051	47.745	47.786	37.339	47.666	52.994	54.928999999999995	37.339	9.127	1.4749999999999999	0.194	30.499	60.328	82.57900000000001	95.074	44.17	50.94
Retrieval	MTEB CQADupstackEnglishRetrieval	30.613	40.781	42.018	42.132999999999996	38.408	46.631	47.332	47.368	38.408	46.379999999999995	50.81	52.663000000000004	38.408	8.656	1.3860000000000001	0.184	30.613	56.44	75.044	86.426	43.766	48.998000000000005
Retrieval	MTEB CQADupstackGamingRetrieval	37.370999999999995	49.718	50.737	50.79	42.884	53.176	53.81700000000001	53.845	42.884	55.826	59.93000000000001	61.013	42.884	9.046999999999999	1.212	0.135	37.370999999999995	70.482	88.425	96.03399999999999	54.43	62.327999999999996
Retrieval	MTEB CQADupstackGisRetrieval	22.875999999999998	31.715	32.847	32.922000000000004	24.52	33.497	34.455000000000005	34.510000000000005	24.52	36.95	42.238	44.147999999999996	24.52	5.9319999999999995	0.901	0.11	22.875999999999998	51.38	75.31099999999999	89.718	36.26	42.248999999999995
Retrieval	MTEB CQADupstackMathematicaRetrieval	14.984	23.457	24.723	24.846	18.159	27.431	28.449	28.52	18.159	28.627999999999997	34.741	37.516	18.159	5.9319999999999995	0.901	0.11	14.984	51.38	75.31099999999999	89.718	36.26	42.248999999999995

Clustering Tasks

Task	Dataset	V-Measure
Clustering	MTEB ArxivClusteringP2P	45.59037428592033
Clustering	MTEB ArxivClusteringS2S	38.86371701986363
Clustering	MTEB BiorxivClusteringP2P	36.551459722989385
Clustering	MTEB BiorxivClusteringS2S	33.69901851846774

Reranking Task

Task	Dataset	MAP	MRR
Reranking	MTEB AskUbuntuDupQuestions	61.625568691427766	75.83256386580486

STS Task

Task	Dataset	Cosine Similarity Pearson	Cosine Similarity Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
STS	MTEB BIOSSES	89.96074355094802	86.2501580394454	82.18427440380462	80.14760935017947	82.24621578156392	80.00363016590163

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご