Bartpho Word

🚀 BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese

BARTpho offers two versions, BARTpho-syllable and BARTpho-word, which are the first publicly available large-scale monolingual sequence-to-sequence models pre-trained for the Vietnamese language. Leveraging the "large" architecture and pre-training scheme of the sequence-to-sequence denoising model BART, BARTpho is well-suited for generative NLP tasks. Experiments on Vietnamese text summarization demonstrate its superiority over the strong baseline mBART in both automatic and human evaluations, advancing the state-of-the-art.

🚀 Quick Start

Two versions of BARTpho, BARTpho-syllable and BARTpho-word, are the first public large - scale monolingual sequence - to - sequence models pre - trained for Vietnamese. They use the "large" architecture and pre - training scheme of the BART sequence - to - sequence denoising model, making them ideal for generative NLP tasks.

✨ Features

• Monolingual for Vietnamese: Specifically pre - trained for the Vietnamese language, catering to the unique linguistic characteristics of Vietnamese.
• Based on BART Architecture: Utilizes the "large" architecture of BART, which is effective for sequence - to - sequence denoising and generative tasks.
• State - of - the - art Performance: Outperforms the strong baseline mBART in Vietnamese text summarization tasks, both in automatic and human evaluations.

📚 Documentation

The general architecture and experimental results of BARTpho can be found in our paper:

@article{bartpho,
title     = {{BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese}},
author    = {Nguyen Luong Tran and Duong Minh Le and Dat Quoc Nguyen},
journal   = {arXiv preprint},
volume    = {arXiv:2109.09701},
year      = {2021}
}

⚠️ Important Note

Please CITE our paper when BARTpho is used to help produce published results or incorporated into other software.

📄 License

This project is licensed under the MIT license.