gte - Qwen2 - 7B - instruct - GGUF Open - source Model: Accurately Calculate Sentence Similarity and Text Embedding

Gte Qwen2 7B Instruct GGUF

Developed by mav23

A 7B-parameter instruction-tuned model based on Qwen2 architecture, specializing in sentence similarity computation and text embedding tasks

Large Language Model Open Source License:Apache-2.0 #Multitask Text Embedding #High Accuracy Classification #Cross-domain Retrieval

Downloads 118

Release Time : 10/10/2024

Model Overview

This model is a 7B-parameter instruction-tuned model developed based on the Qwen2 architecture, primarily used for sentence similarity computation, text embedding, and retrieval tasks. It demonstrates excellent performance on the MTEB benchmark and supports various natural language processing tasks.

Model Features

Excellent Multitask Performance

Outstanding performance across various tasks in the MTEB benchmark, including classification, clustering, and retrieval

Large-scale Parameters

7B parameter scale provides powerful semantic understanding capabilities

Instruction Tuning Optimization

Optimized through instruction tuning, making it particularly suitable for structured tasks

Model Capabilities

Sentence similarity computation

Text classification

Text clustering

Information retrieval

Search result reranking

Semantic text similarity evaluation

Use Cases

E-commerce

Product Review Sentiment Analysis

Analyze sentiment tendencies in Amazon product reviews

Achieved 97.5% accuracy in the AmazonPolarity classification task

Counterfactual Review Detection

Identify counterfactual reviews on Amazon platform

Achieved 91.3% accuracy in the AmazonCounterfactual classification task

Customer Service

Banking Service Classification

Automatically classify bank customer inquiries

Achieved 87.6% accuracy in the Banking77 classification task

Academic Research

Paper Clustering

Topic clustering for arXiv and biorxiv papers

Achieved 56.46 V-measure in the ArxivClusteringP2P task

Q&A Systems

Technical Q&A Retrieval

Retrieve similar questions in technical forums

Achieved 67.58 MAP in the AskUbuntuDupQuestions reranking task

🚀 gte-qwen2-7B-instruct

This is a model related to sentence - similarity tasks, integrating technologies such as mteb, sentence - transformers, transformers, and Qwen2. It has achieved certain performance in various classification, retrieval, clustering, reranking, and STS tasks.

📄 License

This model is licensed under the Apache 2.0 license.

📚 Documentation

Model Information

Property	Details
Model Type	gte - qwen2 - 7B - instruct

Evaluation Results

Classification Tasks

Task	Dataset	Accuracy	AP	F1
Classification	MTEB AmazonCounterfactualClassification (en)	91.31343283582089	67.64251402604096	87.53372530755692
Classification	MTEB AmazonPolarityClassification	97.497825	96.30329547047529	97.49769793778039
Classification	MTEB AmazonReviewsClassification (en)	62.564	N/A	60.975777935041066
Classification	MTEB Banking77Classification	87.56818181818181	N/A	87.25826722019875

Retrieval Tasks

Task	Dataset	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000	Recall@3	Recall@5
Retrieval	MTEB ArguAna	36.486000000000004	54.842	55.206999999999994	55.206999999999994	37.34	55.143	55.509	55.509	36.486000000000004	64.273	65.66199999999999	65.66199999999999	36.486000000000004	9.395000000000001	0.996	0.1	36.486000000000004	93.95400000000001	99.644	99.644	67.283	81.294
Retrieval	MTEB CQADupstackAndroidRetrieval	33.997	48.176	49.82	49.924	42.059999999999995	53.726	54.398	54.416	42.059999999999995	55.574999999999996	60.744	61.85699999999999	42.059999999999995	11.101999999999999	1.73	0.218	33.997	70.35900000000001	91.642	97.977	52.76	61.148
Retrieval	MTEB CQADupstackEnglishRetrieval	35.884	48.14	49.5	49.63	44.458999999999996	53.751000000000005	54.37800000000001	54.415	44.458999999999996	54.157	58.362	60.178	44.458999999999996	10.248	1.5890000000000002	0.207	35.884	64.798	82.345	93.267	51.847	57.601
Retrieval	MTEB CQADupstackGamingRetrieval	39.383	53.714	54.838	54.87800000000001	45.016	56.732000000000006	57.411	57.431	45.016	60.228	64.277	65.07	45.016	9.937	1.288	0.13899999999999998	39.383	76.175	93.02	98.60900000000001	60.265	67.46600000000001
Retrieval	MTEB CQADupstackGisRetrieval	27.426000000000002	37.397000000000006	38.61	38.678000000000004	29.944	39.654	40.638000000000005	40.691	29.944	43.094	48.789	50.339999999999996	29.944	6.78	1.024	0.11800000000000001	27.426000000000002	58.464000000000006	84.193	95.52000000000001	42.172	50.101
Retrieval	MTEB CQADupstackMathematicaRetrieval	19.721	31.604	32.972	33.077	25.0	35.843	36.785000000000004	36.842000000000006	25.0	38.606	44.272	46.527	25.0	N/A	N/A	N/A	19.721	N/A	N/A	N/A	N/A	N/A

Clustering Tasks

Task	Dataset	V - Measure
Clustering	MTEB ArxivClusteringP2P	56.461169803700564
Clustering	MTEB ArxivClusteringS2S	51.73600434466286
Clustering	MTEB BiorxivClusteringP2P	50.09239610327673
Clustering	MTEB BiorxivClusteringS2S	46.64733054606282

Reranking Task

Task	Dataset	MAP	MRR
Reranking	MTEB AskUbuntuDupQuestions	67.57827065898053	79.08136569493911

STS Task

Task	Dataset	Cosine Similarity Pearson	Cosine Similarity Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
STS	MTEB BIOSSES	83.53324575999243	81.37173362822374	82.19243335103444	81.33679307304334	82.38752665975699	81.31510583189689

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご