Crosloengual Bert

🚀 CroSloEngual BERT

A trilingual model based on bert - base architecture, trained on Croatian, Slovenian, and English corpora, offering better performance and cross - lingual knowledge transfer.

🚀 Quick Start

CroSloEngual BERT is a trilingual model that utilizes the bert - base architecture. It has been trained on Croatian, Slovenian, and English corpora. By concentrating on these three languages, this model outperforms the multilingual BERT. Moreover, it provides an option for cross - lingual knowledge transfer, a feature that monolingual models lack.

📚 Documentation

The evaluation of CroSloEngual BERT is presented in our article:

@Inproceedings{ulcar-robnik2020finest,
author = "Ulčar, M. and Robnik-Šikonja, M.",
year = 2020,
title = "{FinEst BERT} and {CroSloEngual BERT}: less is more in multilingual models",
editor = "Sojka, P and Kopeček, I and Pala, K and Horák, A",
booktitle = "Text, Speech, and Dialogue {TSD 2020}",
series = "Lecture Notes in Computer Science",
volume = 12284,
publisher = "Springer",
url = "https://doi.org/10.1007/978-3-030-58323-1_11",
}

The preprint is available at arxiv.org/abs/2006.07890.

📄 License

This project is licensed under the cc - by - 4.0 license.