Llama 2 7b Ukrainian

🚀 Llama-2-7b-Ukrainian

A bilingual pre-trained model supporting Ukrainian and English, offering enhanced text generation capabilities.

🚀 Quick Start

This section is not provided in the original document, so it is skipped.

✨ Features

• Bilingual Support: Llama-2-7b-Ukrainian supports both Ukrainian and English, making it suitable for multilingual text generation tasks.
• Continued Pre-training: It is further pre-trained on a large dataset containing 75% Ukrainian and 25% English documents, enhancing its performance in these languages.

📦 Installation

This section is not provided in the original document, so it is skipped.

💻 Usage Examples

This section is not provided in the original document, so it is skipped.

📚 Documentation

Model Details

Model Description

Llama-2-7b-Ukrainian is a bilingual pre-trained model supporting Ukrainian and English. Continued pre-training from Llama-2-7b on 5B tokens consisting of 75% Ukrainian documents and 25% English documents from CulturaX.

Paper: To Err Is Human, but Llamas Can Learn It Too

Training Hyperparameters

📄 License

The model is under the Llama2 license.

🔧 Technical Details

This section is not provided in the original document, so it is skipped.

📚 Citation

BibTeX:

@article{luhtaru2024err,
  title={To Err Is Human, but Llamas Can Learn It Too},
  author={Luhtaru, Agnes and Purason, Taido and Vainikko, Martin and Del, Maksym and Fishel, Mark},
  journal={arXiv preprint arXiv:2403.05493},
  year={2024}
}