Simcse Roberta Large Zh

🚀 SimCSE(sup)

SimCSE(sup) is a model used for sentence similarity tasks, which can effectively encode sentences into embeddings and calculate the similarity between sentences.

🚀 Quick Start

This project focuses on sentence similarity tasks. It uses multiple Chinese datasets for training and evaluation, and provides a model based on RoBERTa base for sentence encoding and similarity calculation.

✨ Features

• Supports encoding sentences into embeddings.
• Can calculate the cosine similarity between two sentences.

📦 Installation

The original document does not provide installation steps, so this section is skipped.

💻 Usage Examples

Basic Usage

You can use our model for encoding sentences into embeddings.

import torch
from transformers import BertTokenizer
from transformers import BertModel
from sklearn.metrics.pairwise import cosine_similarity

# model
simcse_sup_path = "hellonlp/simcse-roberta-large-zh"
tokenizer = BertTokenizer.from_pretrained(simcse_sup_path)
MODEL = BertModel.from_pretrained(simcse_sup_path)

def get_vector_simcse(sentence):
    """
    预测simcse的语义向量。
    """
    input_ids = torch.tensor(tokenizer.encode(sentence)).unsqueeze(0)
    output = MODEL(input_ids)
    return output.last_hidden_state[:, 0].squeeze(0)

embeddings = get_vector_simcse("武汉是一个美丽的城市。")
print(embeddings.shape)
#torch.Size([1024])

Advanced Usage

You can also compute the cosine similarities between two sentences.

def get_similarity_two(sentence1, sentence2):
    vec1 = get_vector_simcse(sentence1).tolist()
    vec2 = get_vector_simcse(sentence2).tolist()
    similarity_list = cosine_similarity([vec1], [vec2]).tolist()[0][0]
    return similarity_list

sentence1 = '你好吗'
sentence2 = '你还好吗'
result = get_similarity_two(sentence1,sentence2)
print(result)
#0.848331

📚 Documentation

Data List

The following datasets are all in Chinese.

Model List

The evaluation dataset is in Chinese, and we used the same language model RoBERTa base on different methods. In addition, considering that the test set of some datasets is small, which may lead to a large deviation in evaluation accuracy, the evaluation data here uses train, valid and test at the same time, and the final evaluation result adopts the weighted average (w-avg) method.

📄 License

This project is licensed under the MIT license.