Instructor-base Open-source Text Embedding Model - Accurately Calculate Sentence Similarities and Efficiently Complete Text Retrieval

Instructor Base

Developed by hkunlp

A text embedding model based on the T5 architecture, focusing on sentence similarity calculation and text retrieval tasks, with excellent performance in multiple benchmark tests.

Text Embedding

Transformers

EnglishOpen Source License:Apache-2.0 #Text Semantic Similarity #Multi-task Text Embedding #English Information Retrieval

Downloads 13.22k

Release Time : 12/20/2022

Model Overview

This model is a text embedding model based on the T5 architecture, primarily used to generate high-quality sentence embedding vectors, supporting various natural language processing tasks such as information retrieval, text classification, clustering, and semantic similarity calculation.

Model Features

Excellent Multi-task Performance

Performs well in multiple tasks of the MTEB benchmark, including classification, clustering, and retrieval tasks.

Efficient Text Embedding

Capable of generating high-quality sentence embedding vectors, suitable for large-scale information retrieval scenarios.

Broad Applicability

Supports various downstream NLP tasks, including similarity calculation, classification, and clustering.

Model Capabilities

Sentence Similarity Calculation

Text Embedding Generation

Information Retrieval

Text Classification

Text Clustering

Semantic Search

Text Re-ranking

Use Cases

E-commerce

Product Review Classification

Sentiment analysis classification of Amazon product reviews

Achieved 88.36% accuracy in the AmazonPolarity classification task

Counterfactual Detection

Identifying counterfactual statements in Amazon product reviews

Achieved 86.21% accuracy in the AmazonCounterfactual classification task

Finance

Bank Customer Service Classification

Classification of bank customer inquiries

Achieved 77.04% accuracy in the Banking77 classification task

Academic Research

Paper Clustering

Topic clustering of arXiv and biorxiv papers

Achieved a 39.68 v_measure score in the ArxivClusteringP2P task

🚀 Sentence Similarity Model

This model focuses on sentence similarity tasks and is applicable to various text - related scenarios such as information retrieval, text classification, and text clustering.

✨ Features

Multiple Task Support: Capable of handling tasks like Classification, Retrieval, Clustering, Reranking, and STS.
Rich Dataset Adaptability: Tested on a wide range of datasets including MTEB and BeIR datasets.
Comprehensive Metrics: Evaluated using multiple metrics such as accuracy, ap, f1, map, mrr, etc.

📚 Documentation

Model Information

Property	Details
Pipeline Tag	sentence - similarity
Tags	text - embedding, embeddings, information - retrieval, beir, text - classification, language - model, text - clustering, text - semantic - similarity, text - evaluation, prompt - retrieval, text - reranking, sentence - transformers, feature - extraction, sentence - similarity, transformers, t5, English, Sentence Similarity, natural_questions, ms_marco, fever, hotpot_qa, mteb
Language	en
Inference	false
License	apache - 2.0

Model Performance

The model has been tested on multiple datasets with different tasks, and the following are the detailed results:

final_base_results

Task	Dataset	Metrics	Value
Classification	MTEB AmazonCounterfactualClassification (en)	accuracy	86.2089552238806
Classification	MTEB AmazonCounterfactualClassification (en)	ap	55.76273850794966
Classification	MTEB AmazonCounterfactualClassification (en)	f1	81.26104211414781
Classification	MTEB AmazonPolarityClassification	accuracy	88.35995000000001
Classification	MTEB AmazonPolarityClassification	ap	84.18839957309655
Classification	MTEB AmazonPolarityClassification	f1	88.317619250081
Classification	MTEB AmazonReviewsClassification (en)	accuracy	44.64
Classification	MTEB AmazonReviewsClassification (en)	f1	42.48663956478136
Retrieval	MTEB ArguAna	map_at_1	27.383000000000003
Retrieval	MTEB ArguAna	map_at_10	43.024
Retrieval	MTEB ArguAna	map_at_100	44.023
Retrieval	MTEB ArguAna	map_at_1000	44.025999999999996
Retrieval	MTEB ArguAna	map_at_3	37.684
Retrieval	MTEB ArguAna	map_at_5	40.884
Retrieval	MTEB ArguAna	mrr_at_1	28.094
Retrieval	MTEB ArguAna	mrr_at_10	43.315
Retrieval	MTEB ArguAna	mrr_at_100	44.313
Retrieval	MTEB ArguAna	mrr_at_1000	44.317
Retrieval	MTEB ArguAna	mrr_at_3	37.862
Retrieval	MTEB ArguAna	mrr_at_5	41.155
Retrieval	MTEB ArguAna	ndcg_at_1	27.383000000000003
Retrieval	MTEB ArguAna	ndcg_at_10	52.032000000000004
Retrieval	MTEB ArguAna	ndcg_at_100	56.19499999999999
Retrieval	MTEB ArguAna	ndcg_at_1000	56.272
Retrieval	MTEB ArguAna	ndcg_at_3	41.166000000000004
Retrieval	MTEB ArguAna	ndcg_at_5	46.92
Retrieval	MTEB ArguAna	precision_at_1	27.383000000000003
Retrieval	MTEB ArguAna	precision_at_10	8.087
Retrieval	MTEB ArguAna	precision_at_100	0.989
Retrieval	MTEB ArguAna	precision_at_1000	0.099
Retrieval	MTEB ArguAna	precision_at_3	17.093
Retrieval	MTEB ArguAna	precision_at_5	13.044
Retrieval	MTEB ArguAna	recall_at_1	27.383000000000003
Retrieval	MTEB ArguAna	recall_at_10	80.868
Retrieval	MTEB ArguAna	recall_at_100	98.86200000000001
Retrieval	MTEB ArguAna	recall_at_1000	99.431
Retrieval	MTEB ArguAna	recall_at_3	51.28
Retrieval	MTEB ArguAna	recall_at_5	65.22
Clustering	MTEB ArxivClusteringP2P	v_measure	39.68441054431849
Clustering	MTEB ArxivClusteringS2S	v_measure	29.188539728343844
Reranking	MTEB AskUbuntuDupQuestions	map	63.173362687519784
Reranking	MTEB AskUbuntuDupQuestions	mrr	76.18860748362133
STS	MTEB BIOSSES	cos_sim_spearman	82.30789953771232
Classification	MTEB Banking77Classification	accuracy	77.03571428571428
Classification	MTEB Banking77Classification	f1	75.87384305045917
Clustering	MTEB BiorxivClusteringP2P	v_measure	32.98041170516364
Clustering	MTEB BiorxivClusteringS2S	v_measure	25.71652988451154
Retrieval	MTEB CQADupstackAndroidRetrieval	map_at_1	33.739999999999995
Retrieval	MTEB CQADupstackAndroidRetrieval	map_at_10	46.197
Retrieval	MTEB CQADupstackAndroidRetrieval	map_at_100	47.814
Retrieval	MTEB CQADupstackAndroidRetrieval	map_at_1000	47.934
Retrieval	MTEB CQADupstackAndroidRetrieval	map_at_3	43.091
Retrieval	MTEB CQADupstackAndroidRetrieval	map_at_5	44.81
Retrieval	MTEB CQADupstackAndroidRetrieval	mrr_at_1	41.059
Retrieval	MTEB CQADupstackAndroidRetrieval	mrr_at_10	52.292
Retrieval	MTEB CQADupstackAndroidRetrieval	mrr_at_100	52.978
Retrieval	MTEB CQADupstackAndroidRetrieval	mrr_at_1000	53.015
Retrieval	MTEB CQADupstackAndroidRetrieval	mrr_at_3	49.976
Retrieval	MTEB CQADupstackAndroidRetrieval	mrr_at_5	51.449999999999996
Retrieval	MTEB CQADupstackAndroidRetrieval	ndcg_at_1	41.059
Retrieval	MTEB CQADupstackAndroidRetrieval	ndcg_at_10	52.608
Retrieval	MTEB CQADupstackAndroidRetrieval	ndcg_at_100	57.965
Retrieval	MTEB CQADupstackAndroidRetrieval	ndcg_at_1000	59.775999999999996
Retrieval	MTEB CQADupstackAndroidRetrieval	ndcg_at_3	48.473
Retrieval	MTEB CQADupstackAndroidRetrieval	ndcg_at_5	50.407999999999994
Retrieval	MTEB CQADupstackAndroidRetrieval	precision_at_1	41.059
Retrieval	MTEB CQADupstackAndroidRetrieval	precision_at_10	9.943
Retrieval	MTEB CQADupstackAndroidRetrieval	precision_at_100	1.6070000000000002
Retrieval	MTEB CQADupstackAndroidRetrieval	precision_at_1000	0.20500000000000002
Retrieval	MTEB CQADupstackAndroidRetrieval	precision_at_3	23.413999999999998
Retrieval	MTEB CQADupstackAndroidRetrieval	precision_at_5	16.481
Retrieval	MTEB CQADupstackAndroidRetrieval	recall_at_1	33.739999999999995
Retrieval	MTEB CQADupstackAndroidRetrieval	recall_at_10	63.888999999999996
Retrieval	MTEB CQADupstackAndroidRetrieval	recall_at_100	85.832
Retrieval	MTEB CQADupstackAndroidRetrieval	recall_at_1000	97.475
Retrieval	MTEB CQADupstackAndroidRetrieval	recall_at_3	51.953
Retrieval	MTEB CQADupstackAndroidRetrieval	recall_at_5	57.498000000000005
Retrieval	MTEB CQADupstackEnglishRetrieval	map_at_1	31.169999999999998
Retrieval	MTEB CQADupstackEnglishRetrieval	map_at_10	41.455
Retrieval	MTEB CQADupstackEnglishRetrieval	map_at_100	42.716
Retrieval	MTEB CQADupstackEnglishRetrieval	map_at_1000	42.847
Retrieval	MTEB CQADupstackEnglishRetrieval	map_at_3	38.568999999999996
Retrieval	MTEB CQADupstackEnglishRetrieval	map_at_5	40.099000000000004
Retrieval	MTEB CQADupstackEnglishRetrieval	mrr_at_1	39.427
Retrieval	MTEB CQADupstackEnglishRetrieval	mrr_at_10	47.818
Retrieval	MTEB CQADupstackEnglishRetrieval	mrr_at_100	48.519
Retrieval	MTEB CQADupstackEnglishRetrieval	mrr_at_1000	48.558
Retrieval	MTEB CQADupstackEnglishRetrieval	mrr_at_3	45.86
Retrieval	MTEB CQADupstackEnglishRetrieval	mrr_at_5	46.936
Retrieval	MTEB CQADupstackEnglishRetrieval	ndcg_at_1	39.427
Retrieval	MTEB CQADupstackEnglishRetrieval	ndcg_at_10	47.181
Retrieval	MTEB CQADupstackEnglishRetrieval	ndcg_at_100	51.737
Retrieval	MTEB CQADupstackEnglishRetrieval	ndcg_at_1000	53.74
Retrieval	MTEB CQADupstackEnglishRetrieval	ndcg_at_3	43.261
Retrieval	MTEB CQADupstackEnglishRetrieval	ndcg_at_5	44.891
Retrieval	MTEB CQADupstackEnglishRetrieval	precision_at_1	39.427
Retrieval	MTEB CQADupstackEnglishRetrieval	precision_at_10	8.847
Retrieval	MTEB CQADupstackEnglishRetrieval	precision_at_100	1.425
Retrieval	MTEB CQADupstackEnglishRetrieval	precision_at_1000	0.189
Retrieval	MTEB CQADupstackEnglishRetrieval	precision_at_3	20.785999999999998
Retrieval	MTEB CQADupstackEnglishRetrieval	precision_at_5	14.560999999999998
Retrieval	MTEB CQADupstackEnglishRetrieval	recall_at_1	31.169999999999998
Retrieval	MTEB CQADupstackEnglishRetrieval	recall_at_10	56.971000000000004
Retrieval	MTEB CQADupstackEnglishRetrieval	recall_at_100	76.31400000000001
Retrieval	MTEB CQADupstackEnglishRetrieval	recall_at_1000	88.93900000000001
Retrieval	MTEB CQADupstackEnglishRetrieval	recall_at_3	45.208
Retrieval	MTEB CQADupstackEnglishRetrieval	recall_at_5	49.923
Retrieval	MTEB CQADupstackGamingRetrieval	map_at_1	39.682
Retrieval	MTEB CQADupstackGamingRetrieval	map_at_10	52.766000000000005
Retrieval	MTEB CQADupstackGamingRetrieval	map_at_100	53.84100000000001
Retrieval	MTEB CQADupstackGamingRetrieval	map_at_1000	53.898
Retrieval	MTEB CQADupstackGamingRetrieval	map_at_3	49.291000000000004
Retrieval	MTEB CQADupstackGamingRetrieval	map_at_5	51.365
Retrieval	MTEB CQADupstackGamingRetrieval	mrr_at_1	45.266
Retrieval	MTEB CQADupstackGamingRetrieval	mrr_at_10	56.093
Retrieval	MTEB CQADupstackGamingRetrieval	mrr_at_100	56.763
Retrieval	MTEB CQADupstackGamingRetrieval	mrr_at_1000	56.793000000000006
Retrieval	MTEB CQADupstackGamingRetrieval	mrr_at_3	53.668000000000006
Retrieval	MTEB CQADupstackGamingRetrieval	mrr_at_5	55.1
Retrieval	MTEB CQADupstackGamingRetrieval	ndcg_at_1	45.266
Retrieval	MTEB CQADupstackGamingRetrieval	ndcg_at_10	58.836
Retrieval	MTEB CQADupstackGamingRetrieval	ndcg_at_100	62.863
Retrieval	MTEB CQADupstackGamingRetrieval	ndcg_at_1000	63.912
Retrieval	MTEB CQADupstackGamingRetrieval	ndcg_at_3	53.19199999999999
Retrieval	MTEB CQADupstackGamingRetrieval	ndcg_at_5	56.125
Retrieval	MTEB CQADupstackGamingRetrieval	precision_at_1	45.266
Retrieval	MTEB CQADupstackGamingRetrieval	precision_at_10	9.492
Retrieval	MTEB CQADupstackGamingRetrieval	precision_at_100	1.236
Retrieval	MTEB CQADupstackGamingRetrieval	precision_at_1000	0.13699999999999998
Retrieval	MTEB CQADupstackGamingRetrieval	precision_at_3	23.762
Retrieval	MTEB CQADupstackGamingRetrieval	precision_at_5	16.414
Retrieval	MTEB CQADupstackGamingRetrieval	recall_at_1	39.682
Retrieval	MTEB CQADupstackGamingRetrieval	recall_at_10	73.233
Retrieval	MTEB CQADupstackGamingRetrieval	recall_at_100	90.335
Retrieval	MTEB CQADupstackGamingRetrieval	recall_at_1000	97.452
Retrieval	MTEB CQADupstackGamingRetrieval	recall_at_3	58.562000000000005
Retrieval	MTEB CQADupstackGamingRetrieval	recall_at_5	65.569
Retrieval	MTEB CQADupstackGisRetrieval	map_at_1	26.743
Retrieval	MTEB CQADupstackGisRetrieval	map_at_10	34.016000000000005
Retrieval	MTEB CQADupstackGisRetrieval	map_at_100	35.028999999999996
Retrieval	MTEB CQADupstackGisRetrieval	map_at_1000	35.113
Retrieval	MTEB CQADupstackGisRetrieval	map_at_3	31.763
Retrieval	MTEB CQADupstackGisRetrieval	map_at_5	33.013999999999996
Retrieval	MTEB CQADupstackGisRetrieval	mrr_at_1	28.927000000000003
Retrieval	MTEB CQADupstackGisRetrieval	mrr_at_10	36.32
Retrieval	MTEB CQADupstackGisRetrieval	mrr_at_100	37.221
Retrieval	MTEB CQADupstackGisRetrieval	mrr_at_1000	37.281
Retrieval	MTEB CQADupstackGisRetrieval	mrr_at_3	34.105000000000004
Retrieval	MTEB CQADupstackGisRetrieval	mrr_at_5	35.371
Retrieval	MTEB CQADupstackGisRetrieval	ndcg_at_1	28.927000000000003
Retrieval	MTEB CQADupstackGisRetrieval	ndcg_at_10	38.474000000000004
Retrieval	MTEB CQADupstackGisRetrieval	ndcg_at_100	43.580000000000005
Retrieval	MTEB CQADupstackGisRetrieval	ndcg_at_1000	45.64
Retrieval	MTEB CQADupstackGisRetrieval	ndcg_at_3	34.035
Retrieval	MTEB CQADupstackGisRetrieval	ndcg_at_5	36.186
Retrieval	MTEB CQADupstackGisRetrieval	precision_at_1	28.927000000000003
Retrieval	MTEB CQADupstackGisRetrieval	precision_at_10	5.74
Retrieval	MTEB CQADupstackGisRetrieval	precision_at_100	0.8710000000000001
Retrieval	MTEB CQADupstackGisRetrieval	precision_at_1000	0.108
Retrieval	MTEB CQADupstackGisRetrieval	precision_at_3	14.124
Retrieval	MTEB CQADupstackGisRetrieval	precision_at_5	9.74
Retrieval	MTEB CQADupstackGisRetrieval	recall_at_1	26.743
Retrieval	MTEB CQADupstackGisRetrieval	recall_at_10	49.955
Retrieval	MTEB CQADupstackGisRetrieval	recall_at_100	73.904
Retrieval	MTEB CQADupstackGisRetrieval	recall_at_1000	89.133
Retrieval	MTEB CQADupstackGisRetrieval	recall_at_3	38.072
Retrieval	MTEB CQADupstackGisRetrieval	recall_at_5	43.266
Retrieval	MTEB CQADupstackMathematicaRetrieval	map_at_1	16.928
Retrieval	MTEB CQADupstackMathematicaRetrieval	map_at_10	23.549
Retrieval	MTEB CQADupstackMathematicaRetrieval	map_at_100	24.887
Retrieval	MTEB CQADupstackMathematicaRetrieval	map_at_1000	25.018
Retrieval	MTEB CQADupstackMathematicaRetrieval	map_at_3	21.002000000000002
Retrieval	MTEB CQADupstackMathematicaRetrieval	map_at_5	22.256
Retrieval	MTEB CQADupstackMathematicaRetrieval	mrr_at_1	21.02
Retrieval	MTEB CQADupstackMathematicaRetrieval	mrr_at_10	27.898
Retrieval	MTEB CQADupstackMathematicaRetrieval	mrr_at_100	29.018
Retrieval	MTEB CQADupstackMathematicaRetrieval	mrr_at_1000	29.099999999999998
Retrieval	MTEB CQADupstackMathematicaRetrieval	mrr_at_3	25.456
Retrieval	MTEB CQADupstackMathematicaRetrieval	mrr_at_5	26.625
Retrieval	MTEB CQADupstackMathematicaRetrieval	ndcg_at_1	21.02
Retrieval	MTEB CQADupstackMathematicaRetrieval	ndcg_at_10	28.277
Retrieval	MTEB CQADupstackMathematicaRetrieval	ndcg_at_100	34.54
Retrieval	MTEB CQADupstackMathematicaRetrieval	ndcg_at_1000	37.719
Retrieval	MTEB CQADupstackMathematicaRetrieval	ndcg_at_3	23.707
Retrieval	MTEB CQADupstackMathematicaRetrieval	ndcg_at_5	25.482
Retrieval	MTEB CQADupstackMathematicaRetrieval	precision_at_1	21.02
Retrieval	MTEB CQADupstackMathematicaRetrieval	precision_at_10	5.361
Retrieval	MTEB CQADupstackMathematicaRetrieval	precision_at_100	0.9809999999999999
Retrieval	MTEB CQADupstackMathematicaRetrieval	precision_at_1000	0.13899999999999998
Retrieval	MTEB CQADupstackMathematicaRetrieval	precision_at_3	11.401

📄 License

The model is released under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご