🚀 Model Card for EuroLLM-9B-Instruct
This model card provides details about EuroLLM-9B-Instruct, a multilingual large language model. You can also explore its pre-trained version.
🚀 Quick Start
To run the EuroLLM-9B-Instruct model, you can use the following code:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "utter-project/EuroLLM-9B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
messages = [
{
"role": "system",
"content": "You are EuroLLM --- an AI assistant specialized in European languages that provides safe, educational and helpful answers.",
},
{
"role": "user", "content": "What is the capital of Portugal? How would you describe it?"
},
]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
✨ Features
- Multilingual Capability: Capable of understanding and generating text in all European Union languages as well as some additional relevant languages, including Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Irish, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Arabic, Catalan, Chinese, Galician, Hindi, Japanese, Korean, Norwegian, Russian, Turkish, and Ukrainian.
- Instruction Tuned: EuroLLM-9B-Instruct was further instruction tuned on EuroBlocks, an instruction tuning dataset with focus on general instruction-following and machine translation.
📦 Installation
No specific installation steps are provided in the original document.
💻 Usage Examples
Basic Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "utter-project/EuroLLM-9B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
messages = [
{
"role": "system",
"content": "You are EuroLLM --- an AI assistant specialized in European languages that provides safe, educational and helpful answers.",
},
{
"role": "user", "content": "What is the capital of Portugal? How would you describe it?"
},
]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
📚 Documentation
Model Details
The EuroLLM project aims to create a suite of LLMs capable of understanding and generating text in all European Union languages and some additional relevant languages. EuroLLM-9B is a 9B parameter model trained on 4 trillion tokens from various data sources such as web data, parallel data, and high-quality datasets. EuroLLM-9B-Instruct was further instruction tuned on EuroBlocks.
Model Description
EuroLLM uses a standard, dense Transformer architecture with the following features:
- Grouped Query Attention (GQA): With 8 key-value heads to increase inference speed while maintaining downstream performance.
- Pre-layer Normalization: Using RMSNorm for improved training stability and faster computation.
- SwiGLU Activation Function: Shown to lead to good results on downstream tasks.
- Rotary Positional Embeddings (RoPE): Used in every layer to achieve good performance and allow context length extension.
The model was pre-trained using 400 Nvidia H100 GPUs of the Marenostrum 5 supercomputer with a constant batch size of 2,800 sequences (approximately 12 million tokens), the Adam optimizer, and BF16 precision.
Here is a summary of the model hyper-parameters:
Property |
Details |
Sequence Length |
4,096 |
Number of Layers |
42 |
Embedding Size |
4,096 |
FFN Hidden Size |
12,288 |
Number of Heads |
32 |
Number of KV Heads (GQA) |
8 |
Activation Function |
SwiGLU |
Position Encodings |
RoPE (\Theta=10,000) |
Layer Norm |
RMSNorm |
Tied Embeddings |
No |
Embedding Parameters |
0.524B |
LM Head Parameters |
0.524B |
Non-embedding Parameters |
8.105B |
Total Parameters |
9.154B |
Results
EU Languages
Table 1: Comparison of open-weight LLMs on multilingual benchmarks. The borda count corresponds to the average ranking of the models (see (Colombo et al., 2022)). For Arc-challenge, Hellaswag, and MMLU we are using Okapi datasets (Lai et al., 2023) which include 11 languages. For MMLU-Pro and MUSR we translate the English version with Tower (Alves et al., 2024) to 6 EU languages.
* As there are no public versions of the pre-trained models, we evaluated them using the post-trained versions.
The results in Table 1 highlight EuroLLM-9B's superior performance on multilingual tasks compared to other European-developed models (as shown by the Borda count of 1.0), as well as its strong competitiveness with non-European models, achieving results comparable to Gemma-2-9B and outperforming the rest on most benchmarks.
English

Table 2: Comparison of open-weight LLMs on English general benchmarks.
* As there are no public versions of the pre-trained models, we evaluated them using the post-trained versions.
The results in Table 2 demonstrate EuroLLM's strong performance on English tasks, surpassing most European-developed models and matching the performance of Mistral-7B (obtaining the same Borda count).
🔧 Technical Details
The EuroLLM project uses a standard, dense Transformer architecture with specific techniques such as grouped query attention, pre-layer normalization, SwiGLU activation function, and rotary positional embeddings. The model was pre-trained on a large amount of data using a supercomputer with specific hyper-parameters and precision.
📄 License
The model is licensed under the Apache License 2.0.
⚠️ Important Note
EuroLLM-9B has not been aligned to human preferences, so the model may generate problematic outputs (e.g., hallucinations, harmful content, or false statements).