Danskbert

🚀 DanskBERT

DanskBERT is a Danish language model. It addresses the need for high - quality language processing in Danish, offering effective performance on relevant benchmarks.

🚀 Quick Start

Note that you should not prepend the mask with a space when using it directly!

✨ Features

• The model is the best performing base - size model on the ScandEval benchmark for Danish.
• It was trained on the Danish Gigaword Corpus (Strømberg - Derczynski et al., 2021).
• DanskBERT was trained using fairseq with the RoBERTa - base configuration.

🔧 Technical Details

DanskBERT was trained using fairseq with the RoBERTa - base configuration. The model was trained with a batch size of 2k and was trained to convergence for 500k steps using 16 V100 cards for approximately two weeks.

📄 License

This model is licensed under cc - by - 4.0.

📚 Documentation

If you find this model useful, please cite

@inproceedings{snaebjarnarson-etal-2023-transfer,
    title = "{T}ransfer to a Low-Resource Language via Close Relatives: The Case Study on Faroese",
    author = "Snæbjarnarson, Vésteinn  and
      Simonsen, Annika  and
      Glavaš, Goran  and
      Vulić, Ivan",
    booktitle = "Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)",
    month = "may 22--24",
    year = "2023",
    address = "Tórshavn, Faroe Islands",
    publisher = {Link{\"o}ping University Electronic Press, Sweden},
}

⚠️ Important Note

You should not prepend the mask with a space when using it directly!