Italia 9B Instruct V0.1

🚀 Italia 9B - Instruct v0.1

Italia 9B - Instruct v0.1 is an open - source large language model developed by iGenius. It's designed for companies in public and private sectors, with excellent performance in Italian and English, suitable for various NLP tasks.

🚀 Quick Start

For more details on Italia and iGenius, please visit our website and read our release blog post.

Subscribe to our newsletter to receive updates on our latest AI model advancements.

Use with transformers

from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM

model_id = "iGeniusAI/Italia-9B-Instruct-v0.1"

model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)

t_pipeline = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    device_map="auto",
    return_full_text=False, 
    top_p = 0.95, 
    top_k = 50
)

SYSTEM_PROMPT = """Il tuo nome è Modello Italia. Tu sei un'intelligenza artificiale, un modello di linguaggio naturale addestrato da iGenius su Leonardo, uno dei supercomputer più potenti al mondo."""
TEMPERATURE = 0.3
MAX_NEW_TOKENS = 250

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "Ciao come stai?"},
]

conv_template = tokenizer.apply_chat_template(
        messages,
        tokenize=False
    )

outputs = t_pipeline(
    conv_template,
    max_new_tokens=MAX_NEW_TOKENS,
    do_sample=True,
    temperature=TEMPERATURE,
    num_return_sequences=1,
)
print(outputs[0]["generated_text"])

Chat Format

Italy-9B Instruct is a finetuned model to follow instructions provided by a user, so for best results it is necessary to use the chat format as follow:

<|system|>
Your system prompt.</s>
<|user|>
user request.</s>
<|assistant|>

For example:

<|system|>
Il tuo nome è Modello Italia. Tu sei un'intelligenza artificiale, un modello di linguaggio naturale addestrato da iGenius su Leonardo, uno dei supercomputer più potenti al mondo.</s>
<|user|>
Scrivi una funzione python che genera numeri random.</s>
<|assistant|> 

where the model generates the text after <|assistant|>.
</s> is the EOS token.

✨ Features

• Multilingual Proficiency: Italia 9B is proficient in both Italian and English. More than 90% of its pre - training data consists of Italian text, with the remaining portion in English, enabling it to perform well in translation tasks.
• High - Precision Understanding: Trained from scratch in Italian on trillions of tokens, it can understand all Italian linguistic and cultural nuances with unprecedented precision.
• Instruction - Following: The model has undergone a post - training process including supervised fine - tuning and direct preference optimization to enhance instruction - following capabilities.
• Safety Measures: Robust safety measures are ensured through the post - training process.

📦 Installation

The provided code example uses the transformers library. You can install it using the following command:

pip install transformers

📚 Documentation

Introduction

Italia is a family of Open Source large language models developed by iGenius, designed for companies operating in the public and private sectors. The first model in the series, Italia 9B, is a foundational LLM with a 9 - billion - parameter Transformer architecture, developed in collaboration with Cineca and released under the MIT license.

Hardware and Software

Thanks to the partnership with Cineca, Italia 9B was trained and fine - tuned on a large scale using thousands of GPUs on the Leonardo supercomputer, one of the most advanced and high - performing computing infrastructures in the world.

Training

Italia 9B was trained from scratch in Italian on trillions of tokens, using a heterogeneous mix of data: public sources, synthetic data, and domain - specific content provided by commercial partners. More than 90% of the pre - training data is in Italian, and the rest is in English. The model has also undergone a post - training process including supervised fine - tuning and direct preference optimization. The pretraining data has a cutoff date of December 2023.

Benchmarks

All existing benchmarks for evaluating language models are designed for the English - speaking ecosystem. Italia demonstrated nearly state - of - the - art performance among models of a similar size when assessed against benchmarks testing common sense, language understanding, and logical reasoning. Here are the benchmark results generated with llm - harness:

Intended Use

Italia 9B is suitable for both commercial and research purposes, focusing on the Italian language. It can be used for a wide range of natural language processing tasks. However, users should consider aspects such as attribution, limitation of liability, sharing modifications, and compatibility according to the MIT license.

Out of Scope

Italia should not be used for applications related to violations of law, infringement of privacy, malicious activities, misinformation, discriminatory practices, and coding tasks.

Limitations

As a new technology, Italia may produce inaccurate, biased, or otherwise objectionable responses. Developers are recommended to perform safety testing before deploying applications based on it.

Contributors

The iGenius Team. Special thanks to Cineca and their team for their support.

📄 License

Italia 9B is released under the MIT license. The license link is here.

⚠️ Important Note

Italia should not be used for applications related to violations of law, infringement of privacy, malicious activities, misinformation, discriminatory practices, and coding tasks.

💡 Usage Tip

Developers should perform safety testing before deploying any applications based on Italia.