Instructor-xl Open Source Model - Specializing in English Text Semantic Similarity Analysis and Information Retrieval

Instructor Xl

Developed by retrainai

A T5 architecture-based sentence embedding model focused on semantic similarity and information retrieval tasks for English text.

Text Embedding

Transformers

EnglishOpen Source License:Apache-2.0 #Multitask Text Embedding #High-precision Semantic Similarity #Information Retrieval Optimization

Downloads 22

Release Time : 12/28/2023

Model Overview

This model is a T5 architecture-based sentence embedding model primarily used for computing sentence similarity, information retrieval, text classification, and clustering in natural language processing tasks. It performs exceptionally well on multiple standard datasets, particularly in semantic similarity and retrieval tasks.

Model Features

Multitask Performance

Excellent performance across various tasks including sentence similarity, information retrieval, text classification, and clustering.

Powerful Semantic Understanding

Based on the T5 architecture, it deeply understands text semantics and generates high-quality sentence embeddings.

Comprehensive Evaluation

Thoroughly evaluated on multiple standard datasets such as MTEB, validating its effectiveness.

Model Capabilities

Sentence similarity calculation

Information retrieval

Text classification

Text clustering

Feature extraction

Text reranking

Prompt retrieval

Use Cases

Information Retrieval

Question Answering System

Used to retrieve the most relevant answers to user questions.

Achieved map@100 of 38.79 on the CQADupstack dataset.

Document Retrieval

Retrieves relevant content from a large collection of documents.

Achieved ndcg@100 of 58.88 on the ArguAna dataset.

Text Classification

Sentiment Analysis

Classifies text into positive/negative sentiments.

Achieved an accuracy of 86.54% on the AmazonPolarity dataset.

Intent Recognition

Identifies the intent category of user queries.

Achieved an accuracy of 82.66% on the Banking77 dataset.

Semantic Similarity

Duplicate Question Detection

Identifies semantically similar questions.

Achieved a map of 65.35 on the AskUbuntuDupQuestions dataset.

Semantic Search

Search based on semantics rather than keyword matching.

Achieved a Spearman correlation of 84.15 for cosine similarity on the BIOSSES dataset.

🚀 Sentence Similarity Model

This model is designed for sentence similarity tasks. It can be applied in various scenarios such as information retrieval, text classification, and clustering, providing high - quality text embeddings and accurate similarity calculations.

✨ Features

Multiple Task Support: Capable of handling tasks like Classification, Retrieval, Clustering, Reranking, and STS.
Rich Datasets: Tested on a wide range of datasets including MTEB series and BeIR datasets, ensuring its generalization ability.
Comprehensive Metrics: Evaluated with multiple metrics such as accuracy, F1 - score, MAP, MRR, etc., to comprehensively measure the model's performance.

📄 License

This project is licensed under the Apache - 2.0 license.

📚 Documentation

Model Information

Property	Details
Pipeline Tag	sentence - similarity
Tags	text - embedding, embeddings, information - retrieval, beir, text - classification, language - model, text - clustering, text - semantic - similarity, text - evaluation, prompt - retrieval, text - reranking, sentence - transformers, feature - extraction, sentence - similarity, transformers, t5, English, Sentence Similarity, natural_questions, ms_marco, fever, hotpot_qa, mteb
Language	en
Inference	false

Model Results

The model named final_xl_results has the following performance on different tasks and datasets:

Classification Tasks

Dataset	Accuracy	AP	F1
MTEB AmazonCounterfactualClassification (en)	85.08955223880596	52.66066378722476	79.63340218960269
MTEB AmazonPolarityClassification	86.542	81.92695193008987	86.51466132573681
MTEB AmazonReviewsClassification (en)	42.964	-	41.43146249774862
MTEB Banking77Classification	82.66233766233766	-	82.04175284777669

Retrieval Tasks

Dataset	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000
MTEB ArguAna	29.872	46.342	47.152	47.154	30.939	46.756	47.573	47.575	29.872	55.65	58.88099999999999	58.951	29.872	8.549	0.991	0.1	29.872	85.491	99.075	99.644
MTEB CQADupstackAndroidRetrieval	36.797000000000004	48.46	49.968	50.080000000000005	45.494	54.747	55.43599999999999	55.464999999999996	45.494	54.989	60.096000000000004	61.58	45.494	10.558	1.6049999999999998	0.203	36.797000000000004	66.83	88.34100000000001	97.202
MTEB CQADupstackEnglishRetrieval	32.597	43.424	44.78	44.913	40.382	49.219	49.895	49.936	40.382	49.318	53.839999999999996	55.82899999999999	40.382	9.274000000000001	1.497	0.198	32.597	59.882000000000005	78.446	90.88000000000001
MTEB CQADupstackGamingRetrieval	43.8	57.293000000000006	58.321	58.361	49.592000000000006	60.643	61.23499999999999	61.251999999999995	49.592000000000006	63.364	67.167	67.867	49.592000000000006	10.088	1.2930000000000001	0.13899999999999998	43.8	77.635	93.748	98.468
MTEB CQADupstackGisRetrieval	27.721	35.626999999999995	36.719	36.8	29.604999999999997	37.564	38.505	38.565	29.604999999999997	40.575	45.613	47.676	29.604999999999997	6.1690000000000005	0.906	0.11199999999999999	27.721	54.041	76.62299999999999	92.134
MTEB CQADupstackMathematicaRetrieval	16.553	25.384	26.655	26.778000000000002	20.149	29.705	30.672	30.737	20.149	30.843999999999998	36.716	39.495000000000005	20.149	5.858	1.009	0.13799999999999998	16.553	-	-	-

Clustering Tasks

Dataset	V - Measure
MTEB ArxivClusteringP2P	42.452729850641276
MTEB ArxivClusteringS2S	32.21141846480423
MTEB BiorxivClusteringP2P	37.36697339826455
MTEB BiorxivClusteringS2S	30.551241447593092

Reranking Task

Dataset	MAP	MRR
MTEB AskUbuntuDupQuestions	65.34710928952622	77.61124301983028

STS Task

Dataset	Cosine Similarity Spearman
MTEB BIOSSES	84.15312230525639

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご