Mgpt 1.3B Persian

🚀 🇮🇷 Persian mGPT 1.3B

A language model tailored for Persian, boasting 1.3B parameters as the name implies.

Persian, a member of the Indo - European language family, is a highly poetic language spoken by around 110 million people. Here are some key facts:

1. Also known as Farsi, it is predominantly spoken in Iran, Afghanistan, and Tajikistan.
2. It has a rich literary heritage with renowned poets like Rumi, Hafez, and Ferdowsi.
3. It uses the Persian script, a variant of the Arabic script.

🚀 Quick Start

This model is designed to offer high - quality language processing capabilities for Persian text. You can start leveraging its power right away for various NLP tasks.

✨ Features

• Specialized for Persian: Specifically tuned to handle the nuances of the Persian language.
• Derived from a Strong Base: Built upon the robust [mGPT - XL (1.3B)](https://huggingface.co/ai - forever/mGPT) model.

🔧 Technical Details

This model is one of the derivatives of the base [mGPT - XL (1.3B)](https://huggingface.co/ai - forever/mGPT) model. The base model was initially trained on 61 languages from 25 language families using Wikipedia and the C4 corpus.

We sourced additional data for 23 languages, many of which are considered minor, and decided to fine - tune the base model further. Persian mGPT 1.3B was trained for an additional 200 steps with a batch_size of 4 and a context window of 2048 tokens on 1 A100.

The final perplexity of this model on the validation set is 33.44.

Chart of the training loss and perplexity:

📚 Other mGPT - 1.3B Models

• [🇦🇲 mGPT - 1.3B Armenian](https://huggingface.co/ai - forever/mGPT - 1.3B - armenian)
• [🇦🇿 mGPT - 1.3B Azerbaijan](https://huggingface.co/ai - forever/mGPT - 1.3B - azerbaijan)
• [🍯 mGPT - 1.3B Bashkir](https://huggingface.co/ai - forever/mGPT - 1.3B - bashkir)
• [🇧🇾 mGPT - 1.3B Belorussian](https://huggingface.co/ai - forever/mGPT - 1.3B - belorussian)
• [🇧🇬 mGPT - 1.3B Bulgarian](https://huggingface.co/ai - forever/mGPT - 1.3B - bulgarian)
• [🌞 mGPT - 1.3B Buryat](https://huggingface.co/ai - forever/mGPT - 1.3B - buryat)
• [🌳 mGPT - 1.3B Chuvash](https://huggingface.co/ai - forever/mGPT - 1.3B - chuvash)
• [🇬🇪 mGPT - 1.3B Georgian](https://huggingface.co/ai - forever/mGPT - 1.3B - georgian)
• [🌸 mGPT - 1.3B Kalmyk](https://huggingface.co/ai - forever/mGPT - 1.3B - kalmyk)
• [🇰🇿 mGPT - 1.3B Kazakh](https://huggingface.co/ai - forever/mGPT - 1.3B - kazakh)
• [🇰🇬 mGPT - 1.3B Kirgiz](https://huggingface.co/ai - forever/mGPT - 1.3B - kirgiz)
• [🐻 mGPT - 1.3B Mari](https://huggingface.co/ai - forever/mGPT - 1.3B - mari)
• [🇲🇳 mGPT - 1.3B Mongol](https://huggingface.co/ai - forever/mGPT - 1.3B - mongol)
• [🐆 mGPT - 1.3B Ossetian](https://huggingface.co/ai - forever/mGPT - 1.3B - ossetian)
• [🇷🇴 mGPT - 1.3B Romanian](https://huggingface.co/ai - forever/mGPT - 1.3B - romanian)
• [🇹🇯 mGPT - 1.3B Tajik](https://huggingface.co/ai - forever/mGPT - 1.3B - tajik)
• [☕ mGPT - 1.3B Tatar](https://huggingface.co/ai - forever/mGPT - 1.3B - tatar)
• [🇹🇲 mGPT - 1.3B Turkmen](https://huggingface.co/ai - forever/mGPT - 1.3B - turkmen)
• [🐎 mGPT - 1.3B Tuvan](https://huggingface.co/ai - forever/mGPT - 1.3B - tuvan)
• [🇺🇦 mGPT - 1.3B Ukranian](https://huggingface.co/ai - forever/mGPT - 1.3B - ukranian)
• [🇺🇿 mGPT - 1.3B Uzbek](https://huggingface.co/ai - forever/mGPT - 1.3B - uzbek)
• [💎 mGPT - 1.3B Yakut](https://huggingface.co/ai - forever/mGPT - 1.3B - yakut)

📄 License

This project is licensed under the MIT license.

💡 Usage Tip

If you find a bug or have additional data to train the model on your language, please provide us with feedback. The model will be continuously improved over time, so stay tuned!