Scandi Nli Small

🚀 ScandiNLI - Natural Language Inference model for Scandinavian Languages

ScandiNLI is a fine - tuned model based on [jonfd/electra - small - nordic](https://huggingface.co/jonfd/electra - small - nordic), designed for Natural Language Inference in Danish, Norwegian Bokmål, and Swedish. It offers different model sizes to meet various needs.

🚀 Quick Start

You can use this model in your scripts as follows:

Basic Usage

>>> from transformers import pipeline
>>> classifier = pipeline(
...     "zero-shot-classification",
...     model="alexandrainst/scandi-nli-small",
... )
>>> classifier(
...     "Mexicansk bokser advarer Messi - 'Du skal bede til gud, om at jeg ikke finder dig'",
...     candidate_labels=['sundhed', 'politik', 'sport', 'religion'],
...     hypothesis_template="Dette eksempel handler om {}",
... )
{'sequence': "Mexicansk bokser advarer Messi - 'Du skal bede til gud, om at jeg ikke finder dig'",
 'labels': ['religion', 'sport', 'politik', 'sundhed'],
 'scores': [0.4504755437374115,
  0.20737220346927643,
  0.1976872682571411,
  0.14446501433849335]}

✨ Features

• Multilingual Support: Supports Danish, Norwegian Bokmål, and Swedish for Natural Language Inference.
• Multiple Model Sizes: Offers models of different sizes, including large, base, and small, to suit different requirements.

📦 Installation

The original README does not provide installation steps, so this section is skipped.

💻 Usage Examples

The quick - start section already provides a usage example, so no additional example is added here.

📚 Documentation

Performance

We evaluate the models in Danish, Swedish, and Norwegian Bokmål separately, reporting Matthew's Correlation Coefficient (MCC), macro - average F1 - score, and accuracy.

Scandinavian Evaluation

The Scandinavian scores are the average of the Danish, Swedish, and Norwegian scores.

Danish Evaluation

We use a test split of the [DanFEVER dataset](https://aclanthology.org/2021.nodalida - main.pdf#page = 439) to evaluate the Danish performance of the models. The test split is generated using this gist.

Swedish Evaluation

We use the test split of the machine - translated version of the MultiNLI dataset to evaluate the Swedish performance of the models.

Norwegian Evaluation

We use the test split of the machine - translated version of the MultiNLI dataset to evaluate the Norwegian performance of the models.

Training procedure

It has been fine - tuned on a dataset composed of [DanFEVER](https://aclanthology.org/2021.nodalida - main.pdf#page = 439), machine - translated versions of MultiNLI and CommitmentBank into all three languages, and machine - translated versions of [FEVER](https://aclanthology.org/N18 - 1074/) and [Adversarial NLI](https://aclanthology.org/2020.acl - main.441/) into Swedish.

The training split of DanFEVER is generated using this gist. The three languages are sampled equally during training, and they're validated on validation splits of [DanFEVER](https://aclanthology.org/2021.nodalida - main.pdf#page = 439) and machine - translated versions of MultiNLI for Swedish and Norwegian Bokmål, sampled equally.

Check out the Github repository for the code used to train the ScandiNLI models, and the full training logs can be found in [this Weights and Biases report](https://wandb.ai/saattrupdan/huggingface/reports/ScandiNLI--VmlldzozMDQyOTk1?accessToken = r9crgxqvvigy2hatdjeobzwipz7f3id5vqg8ooksljhfw6wl0hv1b05asypsfj9v).

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e - 05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 4242
• gradient_accumulation_steps: 1
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9, 0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• max_steps: 50,000

🔧 Technical Details

The model is fine - tuned on a diverse dataset, including machine - translated versions of multiple NLI datasets. The equal sampling of the three languages during training and validation helps to ensure balanced performance across languages.

📄 License

This project is licensed under the Apache 2.0 license.

Information Table