Bigtranslate

🚀 BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages

This project adapts LLaMA to enhance its multilingual translation ability, covering over 100 languages and showing comparable performance with ChatGPT and Google Translate.

🚀 Quick Start

Large language models (LLMs) have shown promising translation performance across various natural languages. Nevertheless, many LLMs, especially open - sourced ones like BLOOM and LLaMA, are English - dominant and support only dozens of natural languages, leaving the potential of LLMs in language translation largely unexplored.

In this work, we introduce BigTranslate. It adapts LLaMA, which originally covers only 20 languages, and enhances it with multilingual translation capabilities for over 100 languages. BigTranslate is based on LLaMA - 13B and is optimized in three steps:

1. First, we continue training LLaMA with a large amount of Chinese monolingual data.
2. Second, we further train the model using a large - scale parallel dataset covering 102 natural languages.
3. Third, we conduct instruction - tuning on the foundation model with multilingual translation instructions, resulting in the BigTranslate model.

Preliminary experiments on multilingual translation indicate that BigTranslate performs comparably with ChatGPT and Google Translate in many languages and even outperforms ChatGPT in 8 language pairs. We release the BigTranslate model, hoping to promote the research progress.

More Details can be found at https://github.com/ZNLP/BigTranslate and https://arxiv.org/abs/2305.18098

📄 License

The project is licensed under the LGPL - 3.0 license.