Simcse Retromae Small Cs

🚀 SimCSE

SimCSE-RetroMAE-Small is a fine - tuned version of the Seznam/retromae-small-cs model, optimized with the SimCSE objective. This project at Seznam.cz aims to create high - quality, compact Czech semantic embedding models. These models demonstrate excellent performance across a wide range of natural language processing tasks, such as similarity search, retrieval, clustering, and classification. For more detailed information and evaluation results, please refer to the related paper or the GitHub repository.

🚀 Quick Start

You can load and use the model as shown below.

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModel, AutoTokenizer

model_name = "Seznam/retromae-small-cs"  # Hugging Face link
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

input_texts = [
    "Dnes je výborné počasí na procházku po parku.",
    "Večer si oblíbím dobrý film a uvařím si čaj."
]

# Tokenize the input texts
batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')

outputs = model(**batch_dict)
embeddings = outputs.last_hidden_state[:, 0]  # Extract CLS token embeddings

similarity = torch.nn.functional.cosine_similarity(embeddings[0], embeddings[1], dim=0)

📄 License

This model is licensed under the CC - BY - 4.0 license.