Mgpt 1.3B Ukranian

🚀 Ukrainian mGPT 1.3B

This is a language model designed for the Ukrainian language. As the name suggests, the model has 1.3 billion parameters.

Ukrainian belongs to the Indo - European language family. It's a highly melodious language spoken by approximately 40 million people. Here are some key facts about it:

1. It is one of the East Slavic languages, along with Russian and Belarusian.
2. It is the official language of Ukraine and is written using a version of the Cyrillic script.
3. Ukrainian has a rich literary history and maintains a vibrant cultural presence, particularly in poetry and music.

✨ Features

• Tailored for the Ukrainian language.
• Derived from a multi - language base model, offering a solid foundation.
• Further tuned with additional data for better performance.

📚 Documentation

Technical Details

This model is derived from the base [mGPT - XL (1.3B)](https://huggingface.co/ai - forever/mGPT) model (see the list below). The base model was initially trained on 61 languages from 25 language families using Wikipedia and the C4 corpus.

We discovered additional data for 23 languages, most of which are considered minor, and decided to further fine - tune the base model. Ukrainian mGPT 1.3B was trained for an additional 10,000 steps with a batch size of 4 and a context window of 2048 tokens on 1 A100.

The final perplexity of this model on the validation set is 7.1.

Chart of the training loss and perplexity:

Other mGPT - 1.3B models

• [🇦🇲 mGPT - 1.3B Armenian](https://huggingface.co/ai - forever/mGPT - 1.3B - armenian)
• [🇦🇿 mGPT - 1.3B Azerbaijan](https://huggingface.co/ai - forever/mGPT - 1.3B - azerbaijan)
• [🍯 mGPT - 1.3B Bashkir](https://huggingface.co/ai - forever/mGPT - 1.3B - bashkir)
• [🇧🇾 mGPT - 1.3B Belorussian](https://huggingface.co/ai - forever/mGPT - 1.3B - belorussian)
• [🇧🇬 mGPT - 1.3B Bulgarian](https://huggingface.co/ai - forever/mGPT - 1.3B - bulgarian)
• [🌞 mGPT - 1.3B Buryat](https://huggingface.co/ai - forever/mGPT - 1.3B - buryat)
• [🌳 mGPT - 1.3B Chuvash](https://huggingface.co/ai - forever/mGPT - 1.3B - chuvash)
• [🇬🇪 mGPT - 1.3B Georgian](https://huggingface.co/ai - forever/mGPT - 1.3B - georgian)
• [🌸 mGPT - 1.3B Kalmyk](https://huggingface.co/ai - forever/mGPT - 1.3B - kalmyk)
• [🇰🇿 mGPT - 1.3B Kazakh](https://huggingface.co/ai - forever/mGPT - 1.3B - kazakh)
• [🇰🇬 mGPT - 1.3B Kirgiz](https://huggingface.co/ai - forever/mGPT - 1.3B - kirgiz)
• [🐻 mGPT - 1.3B Mari](https://huggingface.co/ai - forever/mGPT - 1.3B - mari)
• [🇲🇳 mGPT - 1.3B Mongol](https://huggingface.co/ai - forever/mGPT - 1.3B - mongol)
• [🐆 mGPT - 1.3B Ossetian](https://huggingface.co/ai - forever/mGPT - 1.3B - ossetian)
• [🇮🇷 mGPT - 1.3B Persian](https://huggingface.co/ai - forever/mGPT - 1.3B - persian)
• [🇷🇴 mGPT - 1.3B Romanian](https://huggingface.co/ai - forever/mGPT - 1.3B - romanian)
• [🇹🇯 mGPT - 1.3B Tajik](https://huggingface.co/ai - forever/mGPT - 1.3B - tajik)
• [☕ mGPT - 1.3B Tatar](https://huggingface.co/ai - forever/mGPT - 1.3B - tatar)
• [🇹🇲 mGPT - 1.3B Turkmen](https://huggingface.co/ai - forever/mGPT - 1.3B - turkmen)
• [🐎 mGPT - 1.3B Tuvan](https://huggingface.co/ai - forever/mGPT - 1.3B - tuvan)
• [🇺🇿 mGPT - 1.3B Uzbek](https://huggingface.co/ai - forever/mGPT - 1.3B - uzbek)
• [💎 mGPT - 1.3B Yakut](https://huggingface.co/ai - forever/mGPT - 1.3B - yakut)

📄 License

This project is licensed under the MIT license.

💡 Usage Tip

If you find a bug or have additional data to train a model for your language, please provide us with feedback.

The model will be improved over time. Stay tuned!