Qwen2 1.5B Ita

🚀 Qwen2 1.5B: Almost the Same Performance as ITALIA (iGenius) but 6 Times Smaller

Qwen2 1.5B is a compact language model fine - tuned for Italian. Despite its small size, it can achieve performance comparable to much larger models, offering a great balance between efficiency and capability for Italian language tasks.

✨ Features

• Compact Size: With only 1.5 billion parameters, it's 6 times smaller than the comparable ITALIA model.
• Strong Performance: Demonstrates high - level performance on various benchmarks, nearly matching larger models.
• Italian - Specific Fine - Tuning: Optimized for Italian language tasks, making it highly effective for Italian language applications.

📦 Installation

The installation steps are not provided in the original README. However, to use the model, you need to have the transformers library installed. You can install it via pip install transformers.

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "DeepMount00/Qwen2-1.5B-Ita"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = [{'role': 'user', 'content': """Marco ha comprato 5 scatole di cioccolatini. Ogni scatola contiene 12 cioccolatini. Ha deciso di dare 3 cioccolatini a ciascuno dei suoi 7 amici. Quanti cioccolatini gli rimarranno dopo averli distribuiti ai suoi amici?"""}]
inputs = tokenizer.apply_chat_template(
    prompt,
    add_generation_prompt=True,
    return_tensors='pt'
)
tokens = model.generate(
    inputs.to(model.device),
    max_new_tokens=1024,
    temperature=0.001,
    do_sample=True
)

print(tokenizer.decode(tokens[0], skip_special_tokens=False))

📚 Documentation

Model Overview

Model Description

Qwen2 1.5B is a compact language model specifically fine - tuned for the Italian language. Despite its relatively small size of 1.5 billion parameters, Qwen2 1.5B demonstrates strong performance, nearly matching the capabilities of larger models, such as the 9 billion parameter ITALIA model by iGenius. The fine - tuning process focused on optimizing the model for various language tasks in Italian, making it highly efficient and effective for Italian language applications.

Performance Evaluation

The performance of Qwen2 1.5B was evaluated on several benchmarks and compared against the ITALIA model. The results are as follows:

📄 License

This project is licensed under the Apache - 2.0 license.