NoInstruct Small Embedding Model v0 Open-Sourced and Launched - Free to Boost Retrieval Task Performance

Noinstruct Small Embedding V0

Developed by avsolatorio

NoInstruct Small Embedding Model v0 is an improved embedding model focused on enhancing retrieval task performance while maintaining independence from arbitrary instruction encoding.

Text Embedding

Transformers

EnglishOpen Source License:MIT #Asymmetric Pooling #Retrieval Optimization #Instruction Independence

Downloads 90.76k

Release Time : 5/1/2024

Model Overview

This model optimizes retrieval performance through an asymmetric pooling strategy: mean pooling for queries and [CLS] representation for sentence/document embeddings, demonstrating superior retrieval performance compared to GIST-small-Embedding-v0.

Model Features

Asymmetric Pooling Strategy

Uses mean pooling for queries and [CLS] representation for sentence/document embeddings to optimize embedding effectiveness in different scenarios.

Instruction Encoding Independence

Maintains independence from arbitrary instruction encoding, aligning with the current popular paradigm for retrieval task embedding models.

Retrieval Performance Optimization

Outperforms the GIST-small-Embedding-v0 model in retrieval tasks.

Model Capabilities

Text Embedding Generation

Semantic Similarity Calculation

Information Retrieval

Use Cases

Information Retrieval

Document Retrieval

Retrieve relevant content from a large corpus of documents based on query statements.

Achieves higher retrieval accuracy compared to GIST-small-Embedding-v0.

Semantic Similarity Calculation

Calculate semantic similarity between different texts.

Obtains more accurate similarity scores through the asymmetric pooling strategy.

🚀 Sentence-Transformers

Sentence-Transformers is a library for computing sentence embeddings and performing sentence similarity tasks. It provides pre-trained models and tools for various NLP tasks.

📚 Documentation

Model Information

Property	Details
Library Name	sentence-transformers
Pipeline Tag	sentence-similarity
Tags	feature-extraction, mteb, sentence-similarity, sentence-transformers, transformers
Model Name	NoInstruct-small-Embedding-v0

Evaluation Results

The model NoInstruct-small-Embedding-v0 has been evaluated on multiple tasks and datasets. Here are the detailed results:

Classification Tasks

Dataset	Accuracy	AP	F1
MTEB AmazonCounterfactualClassification (en)	75.76119402985074	39.03628777559392	69.85860402259618
MTEB AmazonPolarityClassification	93.29920000000001	90.03479490717608	93.28554395248467
MTEB AmazonReviewsClassification (en)	49.98799999999999	-	49.46151232451642
MTEB Banking77Classification	86.3961038961039	-	86.3669961645295

Retrieval Tasks

Dataset	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000
MTEB ArguAna	31.935000000000002	48.791000000000004	49.619	49.623	32.93	49.158	50.00599999999999	50.01	31.935000000000002	57.593	60.841	60.924	31.935000000000002	8.549	0.9900000000000001	0.1	31.935000000000002	85.491	99.004	99.644
MTEB CQADupstackAndroidRetrieval	31.013	42.681999999999995	44.24	44.372	38.196999999999996	48.604	49.315	49.363	38.196999999999996	49.344	54.662	56.665	38.196999999999996	9.571	1.542	0.202	31.013	61.934999999999995	83.923	96.601
MTEB CQADupstackEnglishRetrieval	29.84	39.335	40.647	40.778	36.815	45.175	45.907	45.946999999999996	36.815	44.783	49.551	51.612	36.815	8.363	1.385	0.186	29.84	54.164	74.36	87.484
MTEB CQADupstackGamingRetrieval	39.231	51.44800000000001	52.574	52.629999999999995	44.89	54.803000000000004	55.556000000000004	55.584	44.89	57.228	61.57	62.613	44.89	9.266	1.2309999999999999	0.136	39.231	70.82000000000001	89.446	96.665
MTEB CQADupstackGisRetrieval	25.296000000000003	34.021	35.158	35.233	27.232	36.103	37.076	37.135	27.232	38.878	44.284	46.268	27.232	5.921	0.907	0.11199999999999999	25.296000000000003	51.708	76.36699999999999	91.306
MTEB CQADupstackMathematicaRetrieval	16.24	24.696	25.945	26.069	20.149	29.584	30.548	30.618000000000002	20.149	30.029	35.812	38.755	20.149	4.73	0.74	0.08	16.24	37.24	65.63	86.78

Clustering Tasks

Dataset	V-Measure
MTEB ArxivClusteringP2P	47.78438534940855
MTEB ArxivClusteringS2S	40.12916178519471
MTEB BiorxivClusteringP2P	39.40291404289857
MTEB BiorxivClusteringS2S	35.102356817746816

Reranking Task

Dataset	MAP	MRR
MTEB AskUbuntuDupQuestions	62.125361608299855	74.92525172580574

STS Task

Dataset	Cosine Similarity Pearson	Cosine Similarity Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
MTEB BIOSSES	88.64322910336641	87.20138453306345	87.08547818178234	87.17066094143931	87.30053110771618	86.86824441211934

📄 License

This project is licensed under the MIT License.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご