INSTRUCTOR-large Open-source Text Embedding Model - Free for English Sentence Similarity Calculation and Classification

Instructor Large

Developed by hkunlp

INSTRUCTOR is a text embedding model based on the T5 architecture, focusing on sentence similarity calculation and text classification tasks, and supports English language processing.

Text Embedding

Transformers

EnglishOpen Source License:Apache-2.0 #Text semantic similarity #Multi-task text embedding #Information retrieval optimization

Downloads 186.12k

Release Time : 12/20/2022

Model Overview

This model is mainly used for tasks such as text embedding, sentence similarity calculation, information retrieval, and text classification, and performs excellently in multiple benchmark tests.

Model Features

Multi-task support

Supports multiple text processing tasks, including sentence similarity, text classification, information retrieval, and text clustering.

High performance

Performs excellently on multiple benchmark test datasets, such as the MTEB and BEIR datasets.

Flexible text embedding

Can generate high-quality text embeddings suitable for various downstream tasks.

Model Capabilities

Text embedding

Sentence similarity calculation

Information retrieval

Text classification

Text clustering

Text re-ranking

Feature extraction

Use Cases

E-commerce

Product review classification

Perform sentiment analysis (positive/negative) classification on Amazon product reviews.

Achieved an accuracy of 91.53% on the MTEB AmazonPolarityClassification dataset.

Counterfactual review detection

Identify counterfactual reviews on Amazon.

Achieved an accuracy of 88.13% on the MTEB AmazonCounterfactualClassification dataset.

Customer service

Bank question classification

Classify bank customer questions.

Achieved an accuracy of 78.51% on the MTEB Banking77Classification dataset.

Academic research

Academic paper clustering

Perform topic clustering on academic papers from arXiv and bioRxiv.

Achieved a V-measure of 43.16% on the arXiv paper clustering task.

Question-answering system

Question-answering retrieval

Retrieve relevant questions in a technical Q&A community.

Achieved an average precision of 64.30% on the AskUbuntuDupQuestions dataset.

🚀 INSTRUCTOR

This is a model for sentence similarity, which can be applied in various tasks such as text embedding, information retrieval, text classification, etc.

✨ Features

Multiple Task Support: Suitable for a wide range of NLP tasks including Classification, Retrieval, Clustering, Reranking, and STS.
Multilingual Adaptability: Although mainly focused on English, it can potentially be extended to other languages.
High Performance: Achieves good results on multiple datasets, as shown in the model index.

📚 Documentation

Model Information

Property	Details
Model Type	INSTRUCTOR
Pipeline Tag	sentence - similarity
Tags	text - embedding, embeddings, information - retrieval, beir, text - classification, language - model, text - clustering, text - semantic - similarity, text - evaluation, prompt - retrieval, text - reranking, sentence - transformers, feature - extraction, sentence - similarity, transformers, t5, English, Sentence Similarity, natural_questions, ms_marco, fever, hotpot_qa, mteb
Language	en
Inference	false
License	apache - 2.0

Model Performance Results

The model has been tested on multiple datasets, and here are the performance metrics for different tasks:

Classification Tasks

Task	Dataset	Accuracy	AP	F1
Classification	MTEB AmazonCounterfactualClassification (en)	88.13432835820896	59.298209334395665	83.31769058643586
Classification	MTEB AmazonPolarityClassification	91.526375	88.16327709705504	91.51095801287843
Classification	MTEB AmazonReviewsClassification (en)	47.856	N/A	45.41490917650942
Classification	MTEB Banking77Classification	78.51298701298701	N/A	77.49041754069235

Retrieval Tasks

The retrieval tasks have multiple metrics such as map_at_1, map_at_10, mrr_at_1, etc. Here are some examples for different retrieval datasets:

Task	Dataset	map_at_1	map_at_10	mrr_at_1
Retrieval	MTEB ArguAna	31.223	47.947	32.432
Retrieval	MTEB CQADupstackAndroidRetrieval	35.803000000000004	48.848	44.635000000000005
Retrieval	MTEB CQADupstackEnglishRetrieval	33.706	44.896	41.592

Clustering Tasks

Task	Dataset	V - Measure
Clustering	MTEB ArxivClusteringP2P	43.1621946393635
Clustering	MTEB ArxivClusteringS2S	32.56417132407894

Reranking Tasks

Task	Dataset	MAP	MRR
Reranking	MTEB AskUbuntuDupQuestions	64.29539304390207	76.44484017060196

STS Tasks

Task	Dataset	Cosine Similarity Spearman
STS	MTEB BIOSSES	84.38746499431112

📄 License

This model is licensed under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご