Code Llama 3 8B

🚀 Code-Llama-3-8B

Code-Llama-3-8B is a fully finetuned model trained on massive datasets, excelling in code generation across various programming languages and delivering good performance in math tasks.

✨ Features

• Diverse Datasets: Trained on a refined version of Code-290k-ShareGPT and other datasets like Code-Feedback, orca-math-word-problems-200k, and CodeFeedback-Filtered-Instruction.
• Multilingual Coding: Capable of generating code in multiple languages including Python, Java, JavaScript, GO, C++, Rust, Ruby, Sql, MySql, R, Julia, Haskell, etc.
• Detailed Explanation: Provides detailed explanations/logic behind each generated code.
• Good Math Performance: Produces good outputs in math tasks.

📦 Installation

No installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

This model uses the ChatML prompt format.

<|im_start|>system
You are a helpful Coding assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

You can modify the above prompt as per your requirement. For example:

This is a conversation with your helpful Coding assistant. Assistant can generate Code in various Programming Languages along with necessary explanation.

Example Output

Example 1

Example 2

Example 3

Example 4

Example 5

📚 Documentation

Model Quantization

• GGUF: Link
• Exllama v2: Link

Special Thanks to Bartowski for quantizing this model.

Training Details

• Hardware: Entire dataset was trained on 4 x A100 80GB.
• Duration: For 2 epochs, training took more than 160 hours.
• Codebase: Axolotl & Deepspeed codebase was used for training.
• Base Model: Entire data is trained on Llama-3-8B by Meta.

🔧 Technical Details

The model is fully finetuned and uses the ChatML prompt format. It is trained on a combination of code and math datasets to test its performance in both domains. The massive training datasets contribute to its good results in code generation and math tasks.

📄 License

The model uses the llama3 license.

Information Table

Acknowledgment

The author wants to say special thanks to the Open Source community for helping and guiding to better understand the AI/Model development. Also, thanks for the love and support.