# KoSimCSE-roberta-multitask Open-source Korean Model - Precise Computation of Semantic Similarity

Kosimcse Roberta Multitask

Developed by BM-K

A Korean semantic similarity calculation model optimized based on the RoBERTa architecture, achieving high-performance sentence embeddings through multi-task learning

Text Embedding

Transformers

Korean#Korean Semantic Similarity #Multi-task Learning #High-precision Text Embedding

Downloads 37.37k

Release Time : 6/1/2022

Model Overview

This model is specifically designed for Korean text, capable of encoding sentences into high-dimensional vector space embeddings for calculating semantic similarity between sentences. It supports multiple similarity calculation methods and excels in Korean semantic textual similarity tasks.

Model Features

Multi-task Learning Optimization

Enhances the model's understanding of Korean sentence semantics through multi-task learning strategies

High-performance Similarity Calculation

Achieves an average score of 85.77 in Korean semantic similarity benchmark tests, outperforming similar models

Support for Multiple Similarity Metrics

Supports various similarity calculation methods such as cosine similarity, Euclidean distance, and Manhattan distance

Model Capabilities

Korean Sentence Embedding

Semantic Similarity Calculation

Text Representation Learning

Use Cases

Information Retrieval

🚀 Korean-Sentence-Embedding

This repository focuses on Korean sentence embedding. You can download pre - trained models for immediate inference and also train models in the provided environment.

🚀 Quick Start

import torch
from transformers import AutoModel, AutoTokenizer

def cal_score(a, b):
    if len(a.shape) == 1: a = a.unsqueeze(0)
    if len(b.shape) == 1: b = b.unsqueeze(0)

    a_norm = a / a.norm(dim=1)[:, None]
    b_norm = b / b.norm(dim=1)[:, None]
    return torch.mm(a_norm, b_norm.transpose(0, 1)) * 100

model = AutoModel.from_pretrained('BM-K/KoSimCSE-roberta-multitask') 
AutoTokenizer.from_pretrained('BM-K/KoSimCSE-roberta-multitask')

sentences = ['치타가 들판을 가로 질러 먹이를 쫓는다.',
             '치타 한 마리가 먹이 뒤에서 달리고 있다.',
             '원숭이 한 마리가 드럼을 연주한다.']

inputs = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")
embeddings, _ = model(**inputs, return_dict=False)

score01 = cal_score(embeddings[0][0], embeddings[1][0])
score02 = cal_score(embeddings[0][0], embeddings[2][0])

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModel, AutoTokenizer

def cal_score(a, b):
    if len(a.shape) == 1: a = a.unsqueeze(0)
    if len(b.shape) == 1: b = b.unsqueeze(0)

    a_norm = a / a.norm(dim=1)[:, None]
    b_norm = b / b.norm(dim=1)[:, None]
    return torch.mm(a_norm, b_norm.transpose(0, 1)) * 100

model = AutoModel.from_pretrained('BM-K/KoSimCSE-roberta-multitask') 
AutoTokenizer.from_pretrained('BM-K/KoSimCSE-roberta-multitask')

sentences = ['치타가 들판을 가로 질러 먹이를 쫓는다.',
             '치타 한 마리가 먹이 뒤에서 달리고 있다.',
             '원숭이 한 마리가 드럼을 연주한다.']

inputs = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")
embeddings, _ = model(**inputs, return_dict=False)

score01 = cal_score(embeddings[0][0], embeddings[1][0])
score02 = cal_score(embeddings[0][0], embeddings[2][0])

📚 Documentation

Performance

Semantic Textual Similarity test set results

Property	Details
Model Type	KoSBERT^†_SKT, KoSBERT, KoSRoBERTa, KoSentenceBART, KoSentenceT5, KoSimCSE - BERT^†_SKT, KoSimCSE - BERT, KoSimCSE - RoBERTa, KoSimCSE - BERT - multitask, KoSimCSE - RoBERTa - multitask
AVG	77.40, 80.39, 81.64, 77.14, 77.83, 81.32, 83.37, 83.65, 85.71, 85.77
Cosine Pearson	78.81, 82.13, 81.20, 79.71, 80.87, 82.12, 83.22, 83.60, 85.29, 85.08
Cosine Spearman	78.47, 82.25, 82.20, 78.74, 79.74, 82.56, 83.58, 83.77, 86.02, 86.12
Euclidean Pearson	77.68, 80.67, 81.79, 78.42, 80.24, 81.84, 83.24, 83.54, 85.63, 85.84
Euclidean Spearman	77.78, 80.75, 82.34, 78.02, 79.36, 81.63, 83.60, 83.76, 86.01, 86.12
Manhattan Pearson	77.71, 80.69, 81.59, 78.40, 80.19, 81.99, 83.15, 83.55, 85.57, 85.83
Manhattan Spearman	77.83, 80.78, 82.20, 78.00, 79.27, 81.74, 83.54, 83.77, 85.97, 86.12
Dot Pearson	75.75, 77.96, 80.62, 74.24, 72.81, 79.55, 83.13, 83.55, 85.26, 85.03
Dot Spearman	75.22, 77.90, 81.25, 72.15, 70.17, 79.19, 83.49, 83.64, 85.93, 85.99

Repository Link

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご