Finest Bert

🚀 FinEst BERT

A trilingual model based on the bert - base architecture, trained on Finnish, Estonian, and English corpora, offering better performance and cross - lingual knowledge transfer.

🚀 Quick Start

FinEst BERT is a trilingual model that employs the bert - base architecture. It has been trained on corpora in Finnish, Estonian, and English. By concentrating on these three languages, this model outperforms multilingual BERT. Moreover, it provides an avenue for cross - lingual knowledge transfer, a feature that monolingual models lack.

📚 Documentation

Evaluation

The evaluation of this model is presented in our article:

@Inproceedings{ulcar-robnik2020finest,
author = "Ulčar, M. and Robnik-Šikonja, M.",
year = 2020,
title = "{FinEst BERT} and {CroSloEngual BERT}: less is more in multilingual models",
editor = "Sojka, P and Kopeček, I and Pala, K and Horák, A",
booktitle = "Text, Speech, and Dialogue {TSD 2020}",
series = "Lecture Notes in Computer Science",
volume = 12284,
publisher = "Springer",
url = "https://doi.org/10.1007/978-3-030-58323-1_11",
}

The preprint can be accessed at arxiv.org/abs/2006.07890.

📄 License

This project is licensed under the CC - BY - 4.0 license.