Phogpt 4B

🚀 PhoGPT: Generative Pre-training for Vietnamese

We open-source a cutting - edge generative model series with 4B parameters for the Vietnamese language. It consists of the base pre - trained monolingual model PhoGPT - 4B and its chat - optimized variant, PhoGPT - 4B - Chat. These models outperform previous open - source alternatives.

🚀 Quick Start

We introduce an open - source state - of - the - art generative model series for Vietnamese with 4B parameters. The series includes the base pre - trained monolingual model PhoGPT - 4B and its chat variant, PhoGPT - 4B - Chat.

The base model, PhoGPT - 4B, has precisely 3.7B parameters. It is pre - trained from scratch on a Vietnamese corpus of 102B tokens, with a context length of 8192 and a vocabulary of 20480 token types. The chat variant, PhoGPT - 4B - Chat, is the result of fine - tuning PhoGPT - 4B on a dataset of 70K instructional prompts and their responses, along with an additional 290K conversations.

We've shown that PhoGPT has better performance compared to previous open - source models. You can find more details about the general architecture and experimental results of PhoGPT in our technical report.

@article{PhoGPT,
title     = {{PhoGPT: Generative Pre-training for Vietnamese}},
author    = {Dat Quoc Nguyen and Linh The Nguyen and Chi Tran and Dung Ngoc Nguyen and Dinh Phung and Hung Bui},
journal   = {arXiv preprint},
volume    = {arXiv:2311.02945},
year      = {2023}
}

⚠️ Important Note

Please CITE our technical report when PhoGPT is used to help produce published results or is incorporated into other software.

💡 Usage Tip

For further information or requests, please go to PhoGPT's homepage!

📄 License

The project is licensed under the BSD 3 - Clause license.