Icebert Xlmr Ic3

🚀 IceBERT-xlmr-ic3

This model is trained for Icelandic language tasks, offering capabilities in masked language modeling and more.

🚀 Quick Start

This model was trained with fairseq using the RoBERTa - base architecture. The model xlm - roberta - base was used as a starting point. It is one of many models we have trained for Icelandic. Refer to the paper linked below for more details. The training data used is presented in the table below.

📚 Documentation

Citation

The model is described in this paper https://arxiv.org/abs/2201.05601. Please cite the paper if you make use of the model.

@article{DBLP:journals/corr/abs-2201-05601,
  author    = {V{\'{e}}steinn Sn{\ae}bjarnarson and
               Haukur Barri S{\'{\i}}monarson and
               P{\'{e}}tur Orri Ragnarsson and
               Svanhv{\'{\i}}t Lilja Ing{\'{o}}lfsd{\'{o}}ttir and
               Haukur P{\'{a}}ll J{\'{o}}nsson and
               Vilhj{\'{a}}lmur {\TH}orsteinsson and
               Hafsteinn Einarsson},
  title     = {A Warm Start and a Clean Crawled Corpus - {A} Recipe for Good Language
               Models},
  journal   = {CoRR},
  volume    = {abs/2201.05601},
  year      = {2022},
  url       = {https://arxiv.org/abs/2201.05601},
  eprinttype = {arXiv},
  eprint    = {2201.05601},
  timestamp = {Thu, 20 Jan 2022 14:21:35 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2201-05601.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

📄 License

This model is licensed under cc - by - 4.0.