text2vec-base-multilingual Open Source Text Vectorization Model - Free Support for Multilingual Sentence Similarity Calculation and Feature Extraction

Text2vec Base Multilingual

Developed by barisaydin

A multilingual text vectorization model focused on sentence similarity calculation and feature extraction tasks

Supports Multiple LanguagesOpen Source License:Apache-2.0 #Multilingual Text Embedding #Sentence Similarity Calculation #Cross-Lingual Feature Extraction

Downloads 17

Release Time : 9/20/2023

Model Overview

This model is a multilingual text embedding model capable of converting texts in different languages into vector representations, primarily used for tasks such as sentence similarity calculation, text classification, and clustering. Supports multiple languages including Chinese, English, German, French, and more.

Model Features

Multilingual Support

Supports text vectorization in multiple languages including Chinese, English, German, French, and more

Sentence Similarity Calculation

Specifically optimized for calculating semantic similarity between different sentences

Cross-Lingual Capability

Capable of handling cross-lingual text similarity comparison tasks

Efficient Feature Extraction

Can quickly extract semantic feature representations of texts

Model Capabilities

Text vectorization

Sentence similarity calculation

Cross-lingual text comparison

Text feature extraction

Text classification

Text clustering

Use Cases

E-commerce

Multilingual Review Classification

Classifying product reviews in different languages on Amazon

In the MTEB Amazon multilingual review classification task, English accuracy was 33.13%, Chinese accuracy was 32.52%

Review Polarity Analysis

Analyzing the sentiment polarity (positive/negative) of product reviews

Achieved 66.10% accuracy in the MTEB Amazon polarity classification task

Intent Recognition

Multilingual Intent Classification

Identifying the intent category of user queries

In the MTEB MTOP intent classification task, English accuracy was 62.79%, German was 55.29%

Academic Research

Paper Clustering

Thematic clustering of academic papers

Achieved 32.32% V-measure in the MTEB arXiv paper clustering P2P task

🚀 text2vec-base-multilingual

This is a multilingual model for sentence similarity tasks. It supports multiple languages and has been tested on various datasets, providing a series of performance metrics.

📚 Documentation

Model Information

Property	Details
Pipeline Tag	sentence-similarity
Model Type	text2vec-base-multilingual
Supported Languages	zh, en, de, fr, it, nl, pt, pl, ru
Metrics	spearmanr
Library Name	transformers
Training Data	https://huggingface.co/datasets/shibing624/nli-zh-all/tree/main/text2vec-base-multilingual-dataset

Performance Results

The model has been evaluated on multiple tasks and datasets, here are some of the results:

Classification Tasks

MTEB AmazonCounterfactualClassification:
- (en): Accuracy - 70.97014925373134, AP - 33.95151328318672, F1 - 65.14740155705596
- (de): Accuracy - 68.69379014989293, AP - 79.68277579733802, F1 - 66.54960052336921
- (en-ext): Accuracy - 70.90704647676162, AP - 20.747518928580437, F1 - 58.64365465884924
- (ja): Accuracy - 61.605995717344754, AP - 14.135974879487028, F1 - 49.980224800472136
MTEB AmazonPolarityClassification: Accuracy - 66.103375, AP - 61.10087197664471, F1 - 65.75198509894145
MTEB AmazonReviewsClassification:
- (en): Accuracy - 33.134, F1 - 32.7905397597083
- (de): Accuracy - 33.388, F1 - 33.190561196873084
- (es): Accuracy - 34.824, F1 - 34.297290157740726
- (fr): Accuracy - 33.449999999999996, F1 - 33.08017234412433
- (ja): Accuracy - 30.046, F1 - 29.857141661482228
- (zh): Accuracy - 32.522, F1 - 31.854699911472174
MTEB Banking77Classification: Accuracy - 78.08441558441558, F1 - 77.99825264827898
MTEB EmotionClassification: Accuracy - 43.35, F1 - 38.80269436557695
MTEB ImdbClassification: Accuracy - 59.348, AP - 55.75065220262251, F1 - 58.72117519082607
MTEB MTOPDomainClassification:
- (en): Accuracy - 81.04879160966712, F1 - 80.86889779192701
- (de): Accuracy - 78.59397013243168, F1 - 77.09902761555972
- (es): Accuracy - 79.24282855236824, F1 - 78.75883867079015
- (fr): Accuracy - 76.16661446915127, F1 - 76.30204722831901
- (hi): Accuracy - 78.74506991753317, F1 - 77.50560442779701
- (th): Accuracy - 77.67088607594937, F1 - 77.21442956887493
MTEB MTOPIntentClassification:
- (en): Accuracy - 62.786137710898316, F1 - 46.23474201126368
- (de): Accuracy - 55.285996055226825, F1 - 37.98039513682919
- (es): Accuracy - 58.67911941294196, F1 - 40.541410807124954
- (fr): Accuracy - 53.257124960851854, F1 - 38.42982319259366
- (hi): Accuracy - 59.62352097525995, F1 - 41.28886486568534
- (th): Accuracy - 58.799276672694404, F1 - 43.68379466247341
MTEB MassiveIntentClassification:
- (af): Accuracy - 45.42030934767989, F1 - 44.12201543566376
- (am): Accuracy - 37.67652992602556, F1 - 35.422091900843164
- (ar): Accuracy - 45.02353732347007, F1 - 41.852484084738194
- (az): Accuracy - 48.70880968392737, F1 - 46.904360615435046
- (bn): Accuracy - [Value incomplete in original], F1 - [Value incomplete in original]

Clustering Tasks

MTEB ArxivClusteringP2P: V-measure - 32.31918856561886
MTEB ArxivClusteringS2S: V-measure - 25.503481615956137
MTEB BiorxivClusteringP2P: V-measure - 28.98583420521256
MTEB BiorxivClusteringS2S: V-measure - 23.195091778460892

Reranking Task

MTEB AskUbuntuDupQuestions: MAP - 57.91471462820568, MRR - 71.82990370663501

STS Task

MTEB BIOSSES:
- Cosine Similarity Pearson - 68.83853315193127
- Cosine Similarity Spearman - 66.16174850417771
- Euclidean Pearson - 56.65313897263153
- Euclidean Spearman - 52.69156205876939
- Manhattan Pearson - 56.97282154658304
- Manhattan Spearman - 53.167476517261015

📄 License

This project is licensed under the Apache-2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご