Bloom 1b7 It

🚀 Model Card for Model ID

This model adapts the bloom-1b7 to the Italian language, addressing the lack of Italian support in the original BLOOM model.

🚀 Quick Start

This model is obtained by adapting bloom-1b7 to the Italian language. Among the languages supported by the BLOOM model, there is no Italian, making its use in that context challenging. We adapt the original BLOOM model using the MAD-X language adaptation strategy.

✨ Features

• Adapts the bloom - 1b7 model to the Italian language.
• Utilizes the MAD - X language adaptation strategy.

📚 Documentation

Model Details

We adapt the bloom-1b7 to the Italian language using the MAD-X language adaptation strategy. To produce a valuable model, we follow the same procedure proposed in: https://arxiv.org/abs/2212.09535

We use default script parameters and select a sample of 100,000 examples in the Italian language. We decided to sample data from the Filtered Oscar Dataset for the Italian Language released by Sarti.

It is important to underline that when you use the adapted LLM is necessary to use the tokenizer of the adapted model.

Citation

Pierpaolo Basile, Pierluigi Cassotti, Marco Polignano, Lucia Siciliani, Giovanni Semeraro. On the impact of Language Adaptation for Large Language Models: A case study for the Italian language using only open resources. Proceedings of the Ninth Italian Conference on Computational Linguistics (CLiC-it 2023).

📄 License

BigScience BLOOM RAIL 1.0