Open-source Korean Sentence Embedding Model KoSimCSE-bert-multitask - Generate High-quality Korean Sentence Representations

Kosimcse Bert Multitask

Developed by BM-K

KoSimCSE-BERT-multitask is a high-performance Korean sentence embedding model based on BERT architecture and optimized with multi-task learning strategy, specifically designed for generating high-quality Korean sentence embeddings.

Text Embedding

Transformers

Korean#Korean sentence embedding #Multi-task learning #High-precision semantic matching

Downloads 827

Release Time : 6/1/2022

Model Overview

This model optimizes sentence embeddings through a multi-task learning framework, excelling in Korean semantic similarity tasks and suitable for various Korean natural language processing tasks.

Model Features

Multi-task Learning Optimization

Employs multi-task learning strategy to enhance model performance in semantic similarity tasks

High-performance Korean Understanding

Sentence embedding model specifically optimized for Korean, achieving SOTA in Korean STS tasks

Ready-to-use Pre-trained Model

Provides pre-trained model weights that can be directly downloaded for inference

Model Capabilities

Korean sentence embedding generation

Semantic similarity calculation

Text representation learning

Use Cases

Information Retrieval

🚀 Korean-Sentence-Embedding

This is a Korean sentence embedding repository. You can download pre-trained models and perform inference immediately. It also offers an environment for individuals to train models.

🚀 Quick Start

Here is a quick tour of using the Korean sentence embedding model:

import torch
from transformers import AutoModel, AutoTokenizer

def cal_score(a, b):
    if len(a.shape) == 1: a = a.unsqueeze(0)
    if len(b.shape) == 1: b = b.unsqueeze(0)

    a_norm = a / a.norm(dim=1)[:, None]
    b_norm = b / b.norm(dim=1)[:, None]
    return torch.mm(a_norm, b_norm.transpose(0, 1)) * 100

model = AutoModel.from_pretrained('BM-K/KoSimCSE-bert-multitask') 
AutoTokenizer.from_pretrained('BM-K/KoSimCSE-bert-multitask')

sentences = ['치타가 들판을 가로 질러 먹이를 쫓는다.',
             '치타 한 마리가 먹이 뒤에서 달리고 있다.',
             '원숭이 한 마리가 드럼을 연주한다.']

inputs = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")
embeddings, _ = model(**inputs, return_dict=False)

score01 = cal_score(embeddings[0][0], embeddings[1][0])
score02 = cal_score(embeddings[0][0], embeddings[2][0])

💻 Usage Examples

Basic Usage

The above code in the "Quick tour" section demonstrates the basic usage of loading the model, getting sentence embeddings, and calculating similarity scores.

📊 Performance

Results on the Semantic Textual Similarity test set:

Model	AVG	Cosine Pearson	Cosine Spearman	Euclidean Pearson	Euclidean Spearman	Manhattan Pearson	Manhattan Spearman	Dot Pearson	Dot Spearman
KoSBERT^†_SKT	77.40	78.81	78.47	77.68	77.78	77.71	77.83	75.75	75.22
KoSBERT	80.39	82.13	82.25	80.67	80.75	80.69	80.78	77.96	77.90
KoSRoBERTa	81.64	81.20	82.20	81.79	82.34	81.59	82.20	80.62	81.25

KoSentenceBART	77.14	79.71	78.74	78.42	78.02	78.40	78.00	74.24	72.15
KoSentenceT5	77.83	80.87	79.74	80.24	79.36	80.19	79.27	72.81	70.17

KoSimCSE-BERT^†_SKT	81.32	82.12	82.56	81.84	81.63	81.99	81.74	79.55	79.19
KoSimCSE-BERT	83.37	83.22	83.58	83.24	83.60	83.15	83.54	83.13	83.49
KoSimCSE-RoBERTa	83.65	83.60	83.77	83.54	83.76	83.55	83.77	83.55	83.64

KoSimCSE-BERT-multitask	85.71	85.29	86.02	85.63	86.01	85.57	85.97	85.26	85.93
KoSimCSE-RoBERTa-multitask	85.77	85.08	86.12	85.84	86.12	85.83	86.12	85.03	85.99

You can access the repository at https://github.com/BM-K/Sentence-Embedding-is-all-you-need

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご