Esperberto Small

🚀 EsperBERTo: RoBERTa-like Language model trained on Esperanto

EsperBERTo is a RoBERTa - like language model trained on Esperanto, serving as a companion model to the blog post https://huggingface.co/blog/how-to-train.

🚀 Quick Start

This model is a companion to the blog post How to train a new language model from scratch using Transformers and Tokenizers. You can use it for various NLP tasks related to Esperanto.

📚 Documentation

Training Details

• Current checkpoint: 566000
• Machine name: galinette

💻 Usage Examples

Basic Usage

from transformers import pipeline

fill_mask = pipeline(
    "fill-mask",
    model="julien-c/EsperBERTo-small",
    tokenizer="julien-c/EsperBERTo-small"
)

fill_mask("Jen la komenco de bela <mask>.")

# This is the beginning of a beautiful <mask>.
# =>

# {
#     'score':0.06502299010753632
#     'sequence':'<s> Jen la komenco de bela vivo.</s>'
#     'token':1099
# }
# {
#     'score':0.0421181358397007
#     'sequence':'<s> Jen la komenco de bela vespero.</s>'
#     'token':5100
# }
# {
#     'score':0.024884626269340515
#     'sequence':'<s> Jen la komenco de bela laboro.</s>'
#     'token':1570
# }
# {
#     'score':0.02324388362467289
#     'sequence':'<s> Jen la komenco de bela tago.</s>'
#     'token':1688
# }
# {
#     'score':0.020378097891807556
#     'sequence':'<s> Jen la komenco de bela festo.</s>'
#     'token':4580
# }