Labse En Ru

🚀 LaBSE for English and Russian

This is a truncated version of the LaBSE model, specifically tailored for English and Russian. It offers high - quality sentence embeddings with reduced vocabulary and parameters.

🚀 Quick Start

To get the sentence embeddings using this model, you can use the following code:

import torch
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("cointegrated/LaBSE-en-ru")
model = AutoModel.from_pretrained("cointegrated/LaBSE-en-ru")
sentences = ["Hello World", "Привет Мир"]
encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=64, return_tensors='pt')
with torch.no_grad():
    model_output = model(**encoded_input)
embeddings = model_output.pooler_output
embeddings = torch.nn.functional.normalize(embeddings)
print(embeddings)

✨ Features

• Reduced Vocabulary and Parameters: The current model has only English and Russian tokens left in the vocabulary. The vocabulary is 10% of the original, and the number of parameters in the whole model is 27% of the original, without any loss in the quality of English and Russian embeddings.
• Adaptability: You can adapt the truncation process for other languages, models or datasets, like EIStakovskii/LaBSE-fr-de.

📚 Documentation

This is a truncated version of sentence-transformers/LaBSE, which is, in turn, a port of LaBSE by Google. The model has been truncated in this notebook.

📄 License

The license information can be found at https://tfhub.dev/google/LaBSE/1.

📖 Reference

Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Narveen Ari, Wei Wang. Language-agnostic BERT Sentence Embedding. July 2020