Mt5 Xl

🚀 Google's mT5

A massively multilingual pre - trained text - to - text transformer

mT5 is a multilingual model that addresses the challenges of natural language processing across a wide range of languages. It is pre - trained on a large - scale multilingual corpus, enabling it to perform well on various NLP tasks in different languages.

🚀 Quick Start

To start using mT5, you need to fine - tune it on your specific downstream task as it was only pre - trained on the mC4 corpus without any supervised training.

✨ Features

• Multilingual Support: Covers 101 languages including Afrikaans, Albanian, Amharic, etc.
• Pre - trained on mC4: Utilizes the mC4 corpus for pre - training, which is a large - scale multilingual dataset.
• State - of - the - art Performance: Demonstrates excellent performance on many multilingual benchmarks.

📦 Installation

There is no specific installation steps provided in the original document.

💻 Usage Examples

There is no code example provided in the original document.

📚 Documentation

General Information

• Languages Supported: Afrikaans, Albanian, Amharic, Arabic, Armenian, Azerbaijani, Basque, Belarusian, Bengali, Bulgarian, Burmese, Catalan, Cebuano, Chichewa, Chinese, Corsican, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Gujarati, Haitian Creole, Hausa, Hawaiian, Hebrew, Hindi, Hmong, Hungarian, Icelandic, Igbo, Indonesian, Irish, Italian, Japanese, Javanese, Kannada, Kazakh, Khmer, Korean, Kurdish, Kyrgyz, Lao, Latin, Latvian, Lithuanian, Luxembourgish, Macedonian, Malagasy, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Nepali, Norwegian, Pashto, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Samoan, Scottish Gaelic, Serbian, Shona, Sindhi, Sinhala, Slovak, Slovenian, Somali, Sotho, Spanish, Sundanese, Swahili, Swedish, Tajik, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uzbek, Vietnamese, Welsh, West Frisian, Xhosa, Yiddish, Yoruba, Zulu.
• Pretraining Dataset: mC4
• Other Community Checkpoints: here
• Paper: mT5: A massively multilingual pre - trained text - to - text transformer
• Authors: Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al - Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel

Abstract

The recent "Text - to - Text Transfer Transformer" (T5) leveraged a unified text - to - text format and scale to attain state - of - the - art results on a wide variety of English - language NLP tasks. In this paper, we introduce mT5, a multilingual variant of T5 that was pre - trained on a new Common Crawl - based dataset covering 101 languages. We describe the design and modified training of mT5 and demonstrate its state - of - the - art performance on many multilingual benchmarks. All of the code and model checkpoints used in this work are publicly available.

📄 License

This project is licensed under the Apache - 2.0 license.

⚠️ Important Note

mT5 was only pre - trained on mC4 excluding any supervised training. Therefore, this model has to be fine - tuned before it is useable on a downstream task.