Paraphrase-multilingual-mpnet-base-v2-embedding-all Open-source Model - Free to Calculate Similarity between English and German Sentences for Text Embedding

Paraphrase Multilingual Mpnet Base V2 Embedding All

Developed by LLukas22

This model is a fine-tuned version of paraphrase-multilingual-mpnet-base-v2, supporting English and German sentence similarity calculation, suitable for multilingual text embedding tasks.

Text Embedding

Transformers

Supports Multiple LanguagesOpen Source License:Apache-2.0 #Multilingual sentence embedding #QA system optimization #High-precision semantic matching

Downloads 28

Release Time : 2/23/2023

Model Overview

This model is primarily used for sentence similarity calculation and feature extraction. Fine-tuned on multiple datasets, it can generate high-quality sentence embeddings.

Model Features

Multilingual support

Supports sentence similarity calculation and feature extraction for English and German.

High-quality embeddings

Fine-tuned on multiple high-quality datasets to generate accurate sentence embeddings.

Efficient training

Utilizes mixed-precision training and D-Adaptation optimization techniques for high training efficiency.

Model Capabilities

Sentence similarity calculation

Text feature extraction

Multilingual text processing

Use Cases

Information retrieval

QA system

Can be used to build QA systems for matching questions with relevant answers.

Achieved a top_1 accuracy of 32.4% in evaluation.

Document retrieval

Used to retrieve the most relevant documents for a query.

Achieved a top_10 accuracy of 50.1% in evaluation.

Text analysis

Text clustering

Can be used to cluster semantically similar texts into groups.

🚀 paraphrase-multilingual-mpnet-base-v2-embedding-all

This model is a fine - tuned version of paraphrase - multilingual - mpnet - base - v2, which can effectively handle sentence - similarity tasks and feature extraction.

🚀 Quick Start

This model is a fine - tuned version of paraphrase - multilingual - mpnet - base - v2 on the following datasets: squad, newsqa, LLukas22/cqadupstack, LLukas22/fiqa, LLukas22/scidocs, deepset/germanquad, LLukas22/nq.

📦 Installation

Using this model becomes easy when you have sentence - transformers installed:

pip install -U sentence-transformers

💻 Usage Examples

Basic Usage

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('LLukas22/paraphrase-multilingual-mpnet-base-v2-embedding-all')
embeddings = model.encode(sentences)
print(embeddings)

🔧 Technical Details

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1E+00
per device batch size: 40
effective batch size: 120
seed: 42
optimizer: AdamW with betas (0.9,0.999) and eps 1E - 08
weight decay: 2E - 02
D - Adaptation: True
Warmup: True
number of epochs: 15
mixed_precision_training: bf16

Training results

Epoch	Train Loss	Validation Loss
0	0.085	0.0625
1	0.0598	0.0554
2	0.0484	0.0518
3	0.0405	0.0485
4	0.0341	0.0463
5	0.0287	0.0454
6	0.0243	0.0445
7	0.0207	0.0426
8	0.0177	0.0424
9	0.0153	0.0421
10	0.0134	0.0417
11	0.012	0.0411
12	0.011	0.0414

Evaluation results

Epoch	top_1	top_3	top_5	top_10	top_25
0	0.261	0.351	0.384	0.422	0.459
1	0.272	0.365	0.4	0.439	0.477
2	0.276	0.37	0.404	0.443	0.481
3	0.292	0.391	0.426	0.465	0.503
4	0.295	0.395	0.431	0.47	0.51
5	0.299	0.4	0.437	0.476	0.514
6	0.306	0.404	0.44	0.478	0.515
7	0.309	0.41	0.445	0.485	0.521
8	0.31	0.411	0.448	0.487	0.524
9	0.315	0.417	0.454	0.493	0.529
10	0.319	0.42	0.457	0.495	0.53
11	0.323	0.424	0.46	0.497	0.531
12	0.324	0.427	0.464	0.501	0.536

Framework versions

Transformers: 4.25.1
PyTorch: 2.0.0.dev20230210+cu118
PyTorch Lightning: 1.8.6
Datasets: 2.7.1
Tokenizers: 0.13.1
Sentence Transformers: 2.2.2

📚 Documentation

This model was trained as part of my Master's Thesis 'Evaluation of transformer based language models for use in service information systems'. The source code is available on Github.

📄 License

This model is licensed under the Apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご