Cino Large

🚀 CINO: Pre-trained Language Models for Chinese Minority Languages

CINO is a pre-trained language model designed to empower language understanding in Chinese minority languages. In the field of multilingual pre-trained language models (PLMs), while models like mBERT and XLM - R have shown multilingual and cross - lingual capabilities, there has been a significant gap in the development of PLMs for Chinese minority languages. CINO aims to fill this void, enabling researchers to build more powerful NLP systems.

🚀 Quick Start

To learn more details about CINO, please visit our GitHub repository (in Chinese): [https://github.com/ymcui/Chinese - Minority - PLM](https://github.com/ymcui/Chinese - Minority - PLM)

✨ Features

• Multilingual Support: CINO is trained on multiple Chinese minority languages, including Chinese (zh), Tibetan (bo), Mongolian (mn), Uyghur (ug), Kazakh (kk), Korean (ko), Zhuang, and Cantonese (yue).
• Based on XLM - R: Built on the foundation of XLM - R, CINO undergoes additional pre - training using Chinese minority language corpora, enhancing its performance on these specific languages.

📚 Documentation

Background

Multilingual pre - trained language models such as mBERT and XLM - R have made remarkable progress in providing multilingual and cross - lingual capabilities for language understanding. However, the development of PLMs for Chinese minority languages has been relatively slow, which restricts researchers from creating high - performance NLP systems.

Solution

The Joint Laboratory of HIT and iFLYTEK Research (HFL) proposes CINO, a Chinese - miNOrity pre - trained language model. It is built on XLM - R and further pre - trained with Chinese minority language corpora.

📄 License

This project is licensed under the "apache - 2.0" license.

🔗 Related Projects

You may also be interested in the following projects:

• Chinese MacBERT: https://github.com/ymcui/MacBERT
• Chinese BERT series: [https://github.com/ymcui/Chinese - BERT - wwm](https://github.com/ymcui/Chinese - BERT - wwm)
• Chinese ELECTRA: [https://github.com/ymcui/Chinese - ELECTRA](https://github.com/ymcui/Chinese - ELECTRA)
• Chinese XLNet: [https://github.com/ymcui/Chinese - XLNet](https://github.com/ymcui/Chinese - XLNet)
• Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer
• More resources by HFL: [https://github.com/ymcui/HFL - Anthology](https://github.com/ymcui/HFL - Anthology)