Phobert Large

🚀 PhoBERT: Pre-trained language models for Vietnamese

PhoBERT is a state-of-the-art pre-trained language model tailored for Vietnamese. With "base" and "large" versions, it's the first public large - scale monolingual model for this language. Based on RoBERTa, it optimizes the BERT pre - training process for better performance and outshines previous approaches on multiple Vietnamese NLP tasks.

✨ Features

• First Monolingual Models: The "base" and "large" versions of PhoBERT are the first public large - scale monolingual language models pre - trained for Vietnamese.
• Optimized Pre - training: Its pre - training approach is based on RoBERTa, which optimizes the BERT pre - training procedure for more robust performance.
• State - of - the - art Performance: PhoBERT outperforms previous monolingual and multilingual approaches, achieving new state - of - the - art results on four downstream Vietnamese NLP tasks: Part - of - speech tagging, Dependency parsing, Named - entity recognition, and Natural language inference.

📚 Documentation

The general architecture and experimental results of PhoBERT can be found in our EMNLP - 2020 Findings paper:

@article{phobert,
title     = {{PhoBERT: Pre-trained language models for Vietnamese}},
author    = {Dat Quoc Nguyen and Anh Tuan Nguyen},
journal   = {Findings of EMNLP},
year      = {2020}
}

⚠️ Important Note

Please CITE our paper when PhoBERT is used to help produce published results or is incorporated into other software.

📄 License

This project is licensed under the MIT license.

For further information or requests, please go to PhoBERT's homepage!