SWE Llama 13b

🚀 SWE-Llama

SWE-Llama is a variant of the CodeLlama model, fine - tuned on software engineering tasks from real - world GitHub issues and pull requests, offering enhanced performance in resolving GitHub issues.

🚀 Quick Start

Datasets

• 37 popular Python code repositories
• See princeton - nlp/SWE - bench train split
• See the make_datasets documentation on SWE - bench's GitHub for details on formatting input.

✨ Features

SWE-Llama is fine - tuned on software engineering tasks extracted from real - world GitHub issues and pull requests, aiming to generate patches to resolve GitHub issues based on issue descriptions and code contexts.

📚 Documentation

Model Details

Training Data

SWE-Llama was fine - tuned on 19,000 issues and pull requests collected from 37 popular Python code repositories on GitHub, disjoint from those used in SWE - bench.

Training Procedure

• Fine - tuned only the attention matrices using LoRA method
• Trained for 4 epochs with a batch size of 32
• Selected best checkpoint based on validation perplexity

Evaluation Results

When evaluated on the SWE - bench benchmark:

• SWE - Llama - 7b achieved 3.0% issue resolution rate using oracle context retrieval
• SWE - Llama - 13b achieved 4.0% issue resolution rate using oracle context retrieval

BibTeX Entry

@misc{jimenez2023swebench,
      title={SWE-bench: Can Language Models Resolve Real-World GitHub Issues?}, 
      author={Carlos E. Jimenez and John Yang 
        and Alexander Wettig and Shunyu Yao 
        and Kexin Pei and Ofir Press and Karthik Narasimhan},
      year={2023},
      eprint={2310.06770},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}