gte-Qwen1.5-7B-instruct Open Source Model - Free Computation of Sentence Similarity and Multi-Task Evaluation

Gte Qwen1.5 7B Instruct

Developed by Alibaba-NLP

A 7B-parameter sentence embedding model based on the Qwen1.5 architecture, focusing on sentence similarity calculation and multi-task evaluation

Text Embedding

Transformers

Open Source License:Apache-2.0 #Sentence Similarity Calculation #Multi-task Text Embedding #High-precision Classification

Downloads 253

Release Time : 4/20/2024

Model Overview

This model is a sentence embedding model based on the Qwen1.5 architecture, primarily used for sentence similarity calculation and various natural language processing task evaluations. It performs exceptionally well on the MTEB benchmark, supporting multiple classification, clustering, and retrieval tasks.

Model Features

Multi-task Evaluation Capability

Excellent performance on the MTEB benchmark, supporting various tasks such as classification, clustering, and retrieval

High-performance Sentence Embedding

Capable of generating high-quality sentence embeddings suitable for similarity calculation

Large-scale Parameters

7B parameter scale provides powerful semantic understanding capabilities

Model Capabilities

Sentence similarity calculation

Text classification

Text clustering

Information retrieval

Question answering system support

Semantic search

Use Cases

E-commerce

Product Review Classification

Sentiment polarity classification for Amazon product reviews

Accuracy 96.7%, F1 score 96.69%

Counterfactual Review Detection

Identifying counterfactual reviews on Amazon platform

Accuracy 83.16%, F1 score 77.53%

Academic Research

Paper Clustering

Topic clustering for arXiv and biorxiv papers

V-measure reached 56.4 and 51.45 respectively

Question Answering Systems

Duplicate Question Identification

Identifying duplicate questions in AskUbuntu forum

Mean precision 66.0%, mean reciprocal rank 78.95

🚀 gte-qwen1.5-7b

This is a model that combines multiple techniques and has been tested on various datasets, achieving different performance metrics in tasks such as classification, retrieval, clustering, and reranking.

📚 Documentation

Model Information

Property	Details
Model Type	gte-qwen1.5-7b
Tags	mteb, sentence-transformers, transformers, Qwen, sentence-similarity

Performance Metrics

The model has been evaluated on multiple datasets with different tasks. Here are the detailed results:

Classification Tasks

Dataset	Accuracy	AP	F1
MTEB AmazonCounterfactualClassification (en)	83.16417910447761	49.37655308937739	77.52987230462615
MTEB AmazonPolarityClassification	96.6959	94.90885739242472	96.69477648952649
MTEB AmazonReviewsClassification (en)	62.168	-	60.411431278343755
MTEB Banking77Classification	81.67857142857142	-	80.84103272994895

Retrieval Tasks

Dataset	MAP@1	MAP@10	MAP@100	MAP@1000	MRR@1	MRR@10	MRR@100	MRR@1000	NDCG@1	NDCG@10	NDCG@100	NDCG@1000	Precision@1	Precision@10	Precision@100	Precision@1000	Recall@1	Recall@10	Recall@100	Recall@1000
MTEB ArguAna	36.415	53.505	54.013	54.013	36.842000000000006	53.679	54.17999999999999	54.17999999999999	36.415	62.644999999999996	64.60000000000001	64.60000000000001	36.415	9.161	0.996	0.1	36.415	91.607	99.644	99.644
MTEB CQADupstackAndroidRetrieval	32.694	43.895	45.797	45.922000000000004	40.2	50.11	51.101	51.13100000000001	40.2	50.449999999999996	56.85	58.345	40.2	9.742	1.6480000000000001	0.214	32.694	62.751999999999995	88.619	97.386
MTEB CQADupstackEnglishRetrieval	27.849	37.938	39.211	39.333	34.904	43.869	44.614	44.662	34.904	43.605	48.339999999999996	50.470000000000006	34.904	8.222999999999999	1.332	0.183	27.849	53.635	73.932	87.29599999999999
MTEB CQADupstackGamingRetrieval	29.182999999999996	41.233	42.52	42.589	33.793	44.572	45.456	45.497	33.793	47.823	52.994	54.400000000000006	33.793	8.312999999999999	1.191	0.136	29.182999999999996	64.14999999999999	86.533	96.492
MTEB CQADupstackGisRetrieval	24.389	33.858	35.081	35.161	27.006000000000004	36.378	37.345	37.405	27.006000000000004	39.612	45.216	47.12	27.006000000000004	6.372999999999999	0.968	0.11800000000000001	24.389	55.131	80.315	94.284
MTEB CQADupstackMathematicaRetrieval	15.845999999999998	25.019000000000002	26.478	26.598	20.274	29.221000000000004	30.354999999999997	30.419	20.274	31.014000000000003	37.699	40.363	20.274	6.219	1.101	0.146	15.845999999999998	-	-	-

Clustering Tasks

Dataset	V-Measure
MTEB ArxivClusteringP2P	56.40183100758549
MTEB ArxivClusteringS2S	51.44814171373338
MTEB BiorxivClusteringP2P	49.008657468552016
MTEB BiorxivClusteringS2S	45.05901064421589

Reranking Task

Dataset	MAP	MRR
MTEB AskUbuntuDupQuestions	66.00208703259058	78.95165545442553

STS Task

Dataset	Cosine Similarity Pearson	Cosine Similarity Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman
MTEB BIOSSES	82.12591694410098	81.11570369802254	80.91709076204458	81.11570369802254	80.71719561024605	81.21510355327713

📄 License

The model is released under the Apache-2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご