NV-Embed-v2 Open-source Embedding Model - Suitable for Multiple Natural Language Processing Tasks and Performs Well in Benchmark Tests

NV Embed V2

Developed by nvidia

NV-Embed-v2 is an embedding model based on sentence-transformers, excelling in multiple MTEB benchmarks and suitable for various natural language processing tasks.

Text Embedding

Transformers

English#High-precision text classification #Multi-task embedding #Semantic retrieval optimization

Downloads 35.55k

Release Time : 8/29/2024

Model Overview

This model is primarily used for text embedding and similarity computation, supporting tasks such as classification, clustering, retrieval, and reranking.

Model Features

High-performance text embedding

Excels in multiple MTEB benchmarks, particularly in classification and retrieval tasks.

Multi-task support

Supports various natural language processing tasks, including classification, clustering, retrieval, and reranking.

High accuracy

Achieves 94.28% and 97.74% accuracy in AmazonCounterfactualClassification and AmazonPolarityClassification tasks, respectively.

Model Capabilities

Text classification

Text clustering

Information retrieval

Reranking

Text similarity computation

Use Cases

E-commerce

Product review classification

Used to classify Amazon product reviews, identifying positive and negative feedback.

Achieves 97.74% accuracy in the AmazonPolarityClassification task.

Counterfactual review detection

Detects counterfactual reviews on Amazon to help identify fake or misleading content.

Achieves 94.28% accuracy in the AmazonCounterfactualClassification task.

Academic research

Paper clustering

Clusters academic papers on arXiv and bioRxiv to help researchers discover related studies.

Achieves 55.80% and 54.09% v_measure scores in ArxivClusteringP2P and BiorxivClusteringP2P tasks, respectively.

Technical support

Duplicate question detection

Detects duplicate technical questions in the AskUbuntu community to improve support efficiency.

Achieves 67.46% map score in the AskUbuntuDupQuestions task.

🚀 NV-Embed-v2

NV-Embed-v2 is a model with performance evaluated on various datasets from MTEB. It offers insights into different natural language processing tasks such as classification, retrieval, and clustering.

📚 Documentation

Model Performance on Datasets

Dataset Name	Task Type	Main Score	Other Metrics
MTEB AmazonCounterfactualClassification (en)	Classification	94.28358208955224	accuracy: 94.28358208955224, accuracy_stderr: 0.40076780842082305, ap: 76.49097318319616, ap_stderr: 1.2418692675183929, f1: 91.41982003001168, f1_stderr: 0.5043921413093579
MTEB AmazonPolarityClassification	Classification	97.74185000000001	accuracy: 97.74185000000001, accuracy_stderr: 0.07420471683120942, ap: 96.4737144875525, ap_stderr: 0.2977518241541558, f1: 97.7417581594921, f1_stderr: 0.07428763617010377
MTEB AmazonReviewsClassification (en)	Classification	63.96000000000001	accuracy: 63.96000000000001, accuracy_stderr: 1.815555011559825, f1: 62.49361841640459, f1_stderr: 2.829339314126457
MTEB ArguAna	Retrieval	70.074	map_at_1: 46.515, map_at_10: 62.392, map_at_100: 62.732, map_at_1000: 62.733000000000004, map_at_3: 58.701, map_at_5: 61.027, mrr_at_1: 0.0, mrr_at_10: 0.0, mrr_at_100: 0.0, mrr_at_1000: 0.0, mrr_at_3: 0.0, mrr_at_5: 0.0, ndcg_at_1: 46.515, ndcg_at_10: 70.074, ndcg_at_100: 71.395, ndcg_at_1000: 71.405, ndcg_at_3: 62.643, ndcg_at_5: 66.803, precision_at_1: 46.515, precision_at_10: 9.41, precision_at_100: 0.996, precision_at_1000: 0.1, precision_at_3: 24.68, precision_at_5: 16.814, recall_at_1: 46.515, recall_at_10: 94.097, recall_at_100: 99.57300000000001, recall_at_1000: 99.644, recall_at_3: 74.03999999999999, recall_at_5: 84.068
MTEB ArxivClusteringP2P	Clustering	55.79933795955242	v_measure: 55.79933795955242, v_measure_std: 14.575108141916148
MTEB ArxivClusteringS2S	Clustering	51.262845995850334	v_measure: 51.262845995850334, v_measure_std: 14.727824473104173
MTEB AskUbuntuDupQuestions	Reranking	67.46477327480808	map: 67.46477327480808, mrr: 79.50160488941653
MTEB BIOSSES	STS	87.41644967443246	cosine_pearson: 89.74311007980987, cosine_spearman: 87.41644967443246, manhattan_pearson: 88.57457108347744, manhattan_spearman: 87.59295972042997, euclidean_pearson: 88.27108977118459, euclidean_spearman: 87.41644967443246
MTEB Banking77Classification	Classification	92.41558441558443	accuracy: 92.41558441558443, accuracy_stderr: 0.37701502251934443, f1: 92.38130170447671, f1_stderr: 0.39115151225617767
MTEB BiorxivClusteringP2P	Clustering	54.08649516394218	v_measure: 54.08649516394218, v_measure_std: 0.5303233693045373
MTEB BiorxivClusteringS2S	Clustering	49.60352214167779	v_measure: 49.60352214167779, v_measure_std: 0.7176198612516721
MTEB CQADupstackRetrieval	Retrieval	50.24175000000001	map_at_1: 31.913249999999998, map_at_10: 43.87733333333334, map_at_100: 45.249916666666664, map_at_1000: 45.350583333333326, map_at_3: 40.316833333333335, map_at_5: 42.317083333333336, mrr_at_1: 0.0, mrr_at_10: 0.0, mrr_at_100: 0.0, mrr_at_1000: 0.0, mrr_at_3: 0.0, mrr_at_5: 0.0, ndcg_at_1: 38.30616666666667, ndcg_at_10: 50.24175000000001, ndcg_at_100: 55.345333333333336, ndcg_at_1000: 56.91225000000001, ndcg_at_3: 44.67558333333333, ndcg_at_5: 47.32333333333334, precision_at_1: 38.30616666666667, precision_at_10: 9.007416666666666, precision_at_100: 1.3633333333333333, precision_at_1000: 0.16691666666666666, precision_at_3: 20.895666666666667, precision_at_5: 14.871666666666666, recall_at_1: 31.913249999999998, recall_at_10: 64.11891666666666, recall_at_100: 85.91133333333333, recall_at_1000: 96.28225, recall_at_3: 48.54749999999999, recall_at_5: 55.44283333333334
MTEB ClimateFEVER	Retrieval	45.388	map_at_1: 19.556, map_at_10: 34.623, map_at_100: 36.97, map_at_1000: 37.123, map_at_3: 28.904999999999998, map_at_5: 31.955, mrr_at_1: 0.0, mrr_at_10: 0.0, mrr_at_100: 0.0, mrr_at_1000: 0.0, mrr_at_3: 0.0, mrr_at_5: 0.0, ndcg_at_1: 44.104, ndcg_at_10: 45.388, ndcg_at_100: 52.793, ndcg_at_1000: 55.108999999999995, ndcg_at_3: 38.604, ndcg_at_5: 40.806, precision_at_1: 44.104, precision_at_10: 14.143, precision_at_100: 2.2190000000000003, precision_at_1000: 0.266, precision_at_3: 29.316, precision_at_5: 21.98, recall_at_1: 19.556, recall_at_10: 52.120999999999995, recall_at_100: 76.509, recall_at_1000: 89.029, recall_at_3: 34.919, recall_at_5: 42.18
MTEB DBPedia	Retrieval	53.496	map_at_1: 10.714, map_at_10: 25.814999999999998, map_at_100: 37.845, map_at_1000: 39.974, map_at_3: 17.201, map_at_5: 21.062, mrr_at_1: 0.0, mrr_at_10: 0.0, mrr_at_100: 0.0, mrr_at_1000: 0.0, mrr_at_3: 0.0, mrr_at_5: 0.0, ndcg_at_1: 66.0, ndcg_at_10: 53.496, ndcg_at_100: 58.053, ndcg_at_1000: 64.886, ndcg_at_3: 57.656, ndcg_at_5: 55.900000000000006, precision_at_1: 77.25, precision_at_10: 43.65, precision_at_100: 13.76, precision_at_1000: 2.5940000000000003, precision_at_3: 61.0, precision_at_5: 54.65, recall_at_1: 10.714, recall_at_10: 31.173000000000002, recall_at_100: 63.404, recall_at_1000: 85.874, recall_at_3: 18.249000000000002, recall_at_5: 23.69
MTEB EmotionClassification	Classification	93.38499999999999	accuracy: 93.38499999999999, accuracy_stderr: 0.13793114224133846, f1: 90.12141028353496, f1_stderr: 0.174640257706043
MTEB FEVER	Retrieval	91.52799999999999	map_at_1: 84.66900000000001, map_at_10: 91.52799999999999, map_at_100: 91.721, map_at_1000: 91.73, map_at_3: 90.752, map_at_5: 91.262, mrr_at_1: 0.0, mrr_at_10: 0.0, mrr_at_100: 0.0, mrr_at_1000: 0.0, mrr_at_3: 0.0, mrr_at_5: 0.0, ndcg_at_1: 91.20899999999999, ndcg_at_10: 93.74900000000001, ndcg_at_100: 94.279, ndcg_at_1000: 94.408, ndcg_at_3: 92.923, ndcg_at_5: [Value not fully provided]

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご