Simcse Dist Mpnet Paracrawl Cs En

🚀 SimCSE

SimCSE-DistMPNet-Paracrawl is a fine - tuned model based on [Seznam/dist - mpnet - paracrawl - cs - en](https://huggingface.co/Seznam/dist - mpnet - paracrawl - cs - en) with the SimCSE objective. It is designed to create high - quality small Czech semantic embedding models, which perform well in various NLP tasks.

🚀 Quick Start

SimCSE - DistMPNet - Paracrawl is the result of fine - tuning the [Seznam/dist - mpnet - paracrawl - cs - en](https://huggingface.co/Seznam/dist - mpnet - paracrawl - cs - en) model using the SimCSE objective.

This model was developed at Seznam.cz as part of a project aimed at creating high - quality small Czech semantic embedding models. These models demonstrate excellent performance across a wide range of natural language processing tasks, such as similarity search, retrieval, clustering, and classification. For more detailed information or evaluation results, please refer to the associated paper or [GitHub repository](https://github.com/seznam/czech - semantic - embedding - models).

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModel, AutoTokenizer

model_name = "Seznam/retromae-small-cs"  # Hugging Face link
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

input_texts = [
    "Dnes je výborné počasí na procházku po parku.",
    "Večer si oblíbím dobrý film a uvařím si čaj."
]

# Tokenize the input texts
batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')

outputs = model(**batch_dict)
embeddings = outputs.last_hidden_state[:, 0]  # Extract CLS token embeddings

similarity = torch.nn.functional.cosine_similarity(embeddings[0], embeddings[1], dim=0)

📄 License

This project is licensed under the CC - BY - 4.0 license.