Cino Small V2

🚀 CINO: Pre-trained Language Models for Chinese Minority Languages

CINO is a pre-trained language model designed for Chinese minority languages. It aims to fill the gap in the research of pre-trained models for these languages, enabling researchers to build more powerful NLP systems.

🚀 Quick Start

Multilingual pre-trained language models like mBERT and XLM-R offer multilingual and cross-lingual capabilities for language understanding. In recent years, there has been rapid progress in building multilingual PLMs. However, there is a lack of contributions in building PLMs for Chinese minority languages, which has hindered researchers from developing powerful NLP systems.

To address this issue, the Joint Laboratory of HIT and iFLYTEK Research (HFL) proposes CINO (Chinese-miNOrity pre-trained language model). It is built on XLM-R with additional pre-training using Chinese minority language corpora, including:

• Chinese (zh)
• Tibetan (bo)
• Mongolian (mn)
• Uyghur (ug)
• Kazakh (kk)
• Korean (ko)
• Zhuang
• Cantonese (yue)

For more details (in Chinese), please visit our GitHub repository: CINO GitHub

📚 Documentation

You may also be interested in the following related projects:

• Chinese MacBERT: GitHub
• Chinese BERT series: GitHub
• Chinese ELECTRA: GitHub
• Chinese XLNet: GitHub
• Knowledge Distillation Toolkit - TextBrewer: GitHub

More resources by HFL: HFL Anthology

📄 License

This project is licensed under the Apache-2.0 license.