SGPT-125M Open-Source Sentence Transformer Model - Free Implementation of Sentence Similarity Calculation and Feature Extraction

SGPT 125M Weightedmean Nli Bitfit

Developed by Muennighoff

SGPT-125M is a sentence transformer model based on weighted mean and fine-tuned with Natural Language Inference (NLI), designed for sentence similarity calculation and feature extraction.

Text Embedding

PyTorch

#Multilingual Sentence Similarity #NLI Fine-tuning Optimization #Cross-lingual Text Matching

Downloads 326

Release Time : 3/2/2022

Model Overview

This model is primarily used for sentence similarity calculation and text feature extraction, with optimized multilingual text processing capabilities through weighted mean and NLI fine-tuning.

Model Features

Multi-task Evaluation Capability

Performs well on various tasks in the MTEB (Multi-task Evaluation Benchmark), including classification, clustering, and retrieval.

Multilingual Support

Supports text processing in multiple languages, including English, German, Spanish, French, Japanese, and Chinese.

Weighted Mean Optimization

Uses weighted mean method to optimize sentence representations, improving the accuracy of similarity calculations.

NLI Fine-tuning

Fine-tuned with Natural Language Inference (NLI) tasks to enhance semantic understanding capabilities.

Model Capabilities

Sentence similarity calculation

Text feature extraction

Multilingual text classification

Document clustering

Information retrieval

Search result reranking

Semantic textual similarity evaluation

Bilingual text mining

Use Cases

E-commerce

Amazon Review Classification

Classify multilingual product reviews on Amazon

English review classification accuracy 35.098%, German 24.516%, Spanish 29.098%

Counterfactual Classification

Identify counterfactual statements in Amazon reviews

English accuracy 65.88%, German 59.08%, Japanese 56.42%

Academic Research

arXiv Paper Clustering

Perform point-to-point and sentence-to-sentence clustering on arXiv academic papers

Point-to-point V-measure 34.74, sentence-to-sentence V-measure 24.68

biorxiv Paper Clustering

Cluster analysis on biorxiv biology papers

Point-to-point V-measure 28.93, sentence-to-sentence V-measure 23.08

Q&A Systems

AskUbuntu Duplicate Question Detection

Identify duplicate questions in the AskUbuntu forum

Average precision 52.63%, mean reciprocal rank 65.76%

🚀 SGPT-125M-weightedmean-nli-bitfit

This model is designed for sentence similarity tasks, with capabilities in feature extraction and performance evaluation on multiple datasets.

📚 Documentation

Model Information

Property	Details
Pipeline Tag	sentence-similarity
Tags	sentence-transformers, feature-extraction, sentence-similarity, mteb
Model Name	SGPT-125M-weightedmean-nli-bitfit

Performance Results

Classification Tasks

Dataset	Configuration	Split	Accuracy	AP	F1
MTEB AmazonCounterfactualClassification (en)	en	test	65.88059701492537	28.685493163579785	59.79951005816335
MTEB AmazonCounterfactualClassification (de)	de	test	59.07922912205568	73.91887421019034	56.6316368658711
MTEB AmazonCounterfactualClassification (en-ext)	en-ext	test	64.91754122938531	16.360681214864226	53.126592061523766
MTEB AmazonCounterfactualClassification (ja)	ja	test	56.423982869378996	12.143003571907899	45.76363777987471
MTEB AmazonPolarityClassification	default	test	74.938225	69.58187110320567	74.72744058439321
MTEB AmazonReviewsClassification (en)	en	test	35.098	-	34.73265651435726
MTEB AmazonReviewsClassification (de)	de	test	24.516	-	24.21748200448397
MTEB AmazonReviewsClassification (es)	es	test	29.097999999999995	-	28.620040162757093
MTEB AmazonReviewsClassification (fr)	fr	test	27.395999999999997	-	27.146888644986284
MTEB AmazonReviewsClassification (ja)	ja	test	21.724	-	21.37230564276654
MTEB AmazonReviewsClassification (zh)	zh	test	23.976	-	23.741137981755482
MTEB Banking77Classification	default	test	74.67857142857142	-	74.61743413995573

Retrieval Tasks

Dataset	Configuration	Split	MAP@1	MAP@10	MAP@100	MAP@1000	MAP@3	MAP@5	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	NDCG@3	NDCG@5	Precision@1	Precision@10	Precision@100	Precision@1000	Precision@3	Precision@5	Recall@1	Recall@10	Recall@100	Recall@1000	Recall@3	Recall@5
MTEB ArguAna	default	test	13.442000000000002	24.275	25.588	25.659	20.092	22.439999999999998	13.442000000000002	31.04	37.529	39.348	22.342000000000002	26.595999999999997	13.442000000000002	5.299	0.836	0.098	9.625	7.852	13.442000000000002	52.986999999999995	83.64200000000001	97.795	28.876	39.26
MTEB CQADupstackAndroidRetrieval	default	test	18.221999999999998	24.506	25.611	25.758	22.264999999999997	23.698	23.033	28.719	33.748	37.056	25.240000000000002	27.12	23.033	5.408	1.004	0.158	11.874	8.927	18.221999999999998	36.355	58.724	81.33500000000001	26.334000000000003	31.4
MTEB CQADupstackEnglishRetrieval	default	test	12.058	16.051000000000002	16.772000000000002	16.871	14.78	15.5	15.35	18.804000000000002	22.346	25.007	16.768	17.692	15.35	3.51	0.664	0.11100000000000002	7.983	5.656	12.058	23.644000000000002	39.76	58.56	17.541999999999998	20.232
MTEB CQADupstackGamingRetrieval	default	test	21.183	28.9	29.858	29.953999999999997	26.58	27.912	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-

Clustering Tasks

Dataset	Configuration	Split	V-Measure
MTEB ArxivClusteringP2P	default	test	34.742482477870766
MTEB ArxivClusteringS2S	default	test	24.67870651472156
MTEB BlurbsClusteringS2S	default	test	8.00311862863495
MTEB BiorxivClusteringP2P	default	test	28.93427045246491
MTEB BiorxivClusteringS2S	default	test	23.080939123955474

Reranking Task

Dataset	Configuration	Split	MAP	MRR
MTEB AskUbuntuDupQuestions	default	test	52.63439984994702	65.75704612408214

STS Task

Dataset	Configuration	Split	Cos Sim Pearson	Cos Sim Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
MTEB BIOSSES	default	test	72.78000135012542	70.92812216947605	77.1169214949292	77.10175681583313	76.84527031837595	77.0704308008438

Bitext Mining Tasks

Dataset	Configuration	Split	Accuracy	F1	Precision	Recall
MTEB BUCC (de-en)	de-en	test	1.0960334029227559	1.0925539318023658	1.0908141962421711	1.0960334029227559
MTEB BUCC (fr-en)	fr-en	test	0.02201188641866608	0.02201188641866608	0.02201188641866608	0.02201188641866608
MTEB BUCC (ru-en)	ru-en	test	0.0	0.0	0.0	0.0
MTEB BUCC (zh-en)	zh-en	test	0.0	0.0	0.0	0.0

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご