Codellama 34b Python Hf

🚀 Code Llama

Code Llama is a collection of pretrained and fine - tuned generative text models with parameter scales ranging from 7 billion to 34 billion. This repository hosts the 34B Python specialist version in the Hugging Face Transformers format. The model is designed for general code synthesis and understanding. You can find links to other models in the index at the bottom.

🚀 Quick Start

To use this model, please make sure to install transformers:

pip install transformers accelerate

✨ Features

• Code Completion: The model can assist in completing code snippets.
• Python Specialist: Specifically optimized for Python programming.

📚 Documentation

Model Details

• Model Developers: Meta
• Variations: Code Llama comes in three model sizes (7B, 13B, 34B, 70B) and three variants:
  • Code Llama: Base models for general code synthesis and understanding.
  • Code Llama - Python: Designed specifically for Python.
  • Code Llama - Instruct: For instruction following and safer deployment.
• This Repository: Contains the Python version of the 34B parameters model.
• Input: The models accept text input only.
• Output: The models generate text only.
• Model Architecture: Code Llama is an auto - regressive language model using an optimized transformer architecture.
• Model Dates: Trained between January 2023 and July 2023.
• Status: A static model trained on an offline dataset. Future versions of Code Llama - Instruct will be released as model safety is improved with community feedback.

Intended Use

• Intended Use Cases: For commercial and research use in English and relevant programming languages. The base model can be adapted for various code synthesis and understanding tasks, Code Llama - Python is for Python programming, and Code Llama - Instruct is safer for code assistant and generation applications.
• Out - of - Scope Uses: Any use that violates applicable laws or regulations (including trade compliance laws), use in languages other than English, and any use prohibited by the Acceptable Use Policy and Licensing Agreement.

Hardware and Software

• Training Factors: Custom training libraries were used. Training and fine - tuning of the released models were performed on Meta’s Research Super Cluster.
• Carbon Footprint: Training all 9 Code Llama models required 400K GPU hours of computation on A100 - 80GB hardware (TDP of 350 - 400W). Estimated total emissions were 65.3 tCO2eq, 100% offset by Meta’s sustainability program.

Training Data

All experiments and released models were trained and fine - tuned using the same data as Llama 2 with different weights. See Section 2 and Table 1 in the research paper for details.

Evaluation Results

See evaluations for the main models and detailed ablations in Section 3 and safety evaluations in Section 4 of the research paper.

Ethical Considerations and Limitations

Code Llama and its variants are new technologies with risks. Testing so far has been in English and cannot cover all scenarios. The model's potential outputs cannot be predicted in advance, and it may produce inaccurate or objectionable responses. Developers should perform safety testing and tuning for specific applications before deployment. Refer to the Responsible Use Guide.

📄 License

A custom commercial license is available at: https://ai.meta.com/resources/models-and-libraries/llama-downloads/

Model Comparison Table

⚠️ Important Note

This is a non - official Code Llama repo. You can find the official Meta repository in the Meta Llama organization.