Nordic - RoBERTa - Wiki Open - source Model - Supports Masked Language Modeling Tasks for Nordic Languages like Swedish

Nordic Roberta Wiki

Developed by flax-community

A Nordic RoBERTa model trained on Swedish, Danish, and Norwegian Wikipedia, suitable for fill-mask tasks.

Large Language Model Other#Nordic Multilingual Model #Wikipedia Training #Fill-Mask Task

Downloads 13

Release Time : 3/2/2022

Model Overview

This model is a RoBERTa model trained on Swedish, Danish, and Norwegian Wikipedia, primarily used for fill-mask tasks and supports multilingual processing for Nordic languages.

Model Features

Multilingual Support

Supports three Nordic languages: Swedish, Danish, and Norwegian.

Wikipedia-Based Training

Training data sourced from Swedish, Danish, and Norwegian Wikipedia content.

High Performance

Excellent performance in Danish named entity recognition and sentiment analysis tasks.

Model Capabilities

Fill-Mask

Multilingual Text Processing

Named Entity Recognition

Sentiment Analysis

Use Cases

Natural Language Processing

Named Entity Recognition

Used to identify named entities in text, such as person names, locations, etc.

Achieved an average F1 score of 85.75 on the DaNE dataset.

Sentiment Analysis

Used to analyze the sentiment tendency of text.

Achieved an average score of 56.46 in sentiment analysis tasks.

🚀 Nordic Roberta Wikipedia

A Nordic RoBERTa model trained on Swedish, Danish, and Norwegian Wikipedia, designed for fill - mask tasks.

🚀 Quick Start

This model can be used for fill - mask tasks. You can use it through the Hugging Face Transformers library.

✨ Features

Trained on Swedish, Danish, and Norwegian Wikipedia, which can better adapt to Nordic languages.
Suitable for multiple NLP tasks such as named entity recognition and sentiment analysis.

📚 Documentation

Description

The Nordic RoBERTa model is trained on the Swedish, Danish, and Norwegian Wikipedia.

Evaluation

Named Entity Recognition in Danish

I fine - tuned each model for 3 epochs on DaNE, repeated it 5 times for each model, and calculated 95% confidence intervals for the means. Here are the results:

xlm - roberta - base: 88.01 ± 0.43
flax - community/nordic - roberta - wiki: 85.75 ± 0.69 (this model)
Maltehb/danish - bert - botxo: 85.38 ± 0.55
flax - community/roberta - base - danish: 80.14 ± 1.47
flax - community/roberta - base - scandinavian: 78.03 ± 3.02
Maltehb/-l - ctra - danish - electra - small - cased: 57.87 ± 3.19
NbAiLab/nb - bert - base: 30.24 ± 1.21
Randomly initialised RoBERTa model: 19.79 ± 2.00

Sentiment Analysis in Danish

Here are the results on the test set, where each model has been trained 5 times, and the “±” refers to a 95% confidence interval of the mean score:

Maltehb/danish - bert - botxo: 65.19 ± 0.53
NbAiLab/nb - bert - base: 63.80 ± 0.77
xlm - roberta - base: 63.55 ± 1.59
flax - community/nordic - roberta - wiki: 56.46 ± 1.77
flax - community/roberta - base - danish: 54.73 ± 8.96
flax - community/roberta - base - scandinavian: 44.28 ± 9.21
Maltehb/-l - ctra - danish - electra - small - cased: 47.78 ± 12.65
Randomly initialised RoBERTa model: 36.96 ± 1.02
Maltehb/roberta - base - scandinavian: 33.65 ± 8.32

Model Series

This model is part of a series of models trained on TPU with Flax Jax during the Huggingface Flax/Jax challenge.

Related Models

GPT Models

Swedish GPT: [https://huggingface.co/birgermoell/swedish - gpt/](https://huggingface.co/birgermoell/swedish - gpt/)
Swedish gpt wiki: [https://huggingface.co/flax - community/swe - gpt - wiki](https://huggingface.co/flax - community/swe - gpt - wiki)
Nordic gpt wiki: [https://huggingface.co/flax - community/nordic - gpt - wiki](https://huggingface.co/flax - community/nordic - gpt - wiki)
Dansk gpt wiki: [https://huggingface.co/flax - community/dansk - gpt - wiki](https://huggingface.co/flax - community/dansk - gpt - wiki)
Norsk gpt wiki: [https://huggingface.co/flax - community/norsk - gpt - wiki](https://huggingface.co/flax - community/norsk - gpt - wiki)

RoBERTa Models

Nordic Roberta Wiki: [https://huggingface.co/flax - community/nordic - roberta - wiki](https://huggingface.co/flax - community/nordic - roberta - wiki)
Swe Roberta Wiki Oscar: [https://huggingface.co/flax - community/swe - roberta - wiki - oscar](https://huggingface.co/flax - community/swe - roberta - wiki - oscar)
Roberta Swedish Scandi: [https://huggingface.co/birgermoell/roberta - swedish - scandi](https://huggingface.co/birgermoell/roberta - swedish - scandi)
Roberta Swedish: [https://huggingface.co/birgermoell/roberta - swedish](https://huggingface.co/birgermoell/roberta - swedish)

Swedish T5 Model

Swedish T5 model: [https://huggingface.co/birgermoell/t5 - base - swedish](https://huggingface.co/birgermoell/t5 - base - swedish)

📄 License

This model is licensed under CC - BY - 4.0.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご