Codefuse 13B

🚀 CodeFuse-13B

CodeFuse-13B is a 13-billion parameter code generation model trained on the GPT-NeoX framework, capable of handling code sequences up to 4096 characters. It can generate accurate, efficient, and compliant code.

[中文] [English]

🚀 Quick Start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(("CodeFuse-13B"))
model = AutoModelForCausalLM.from_pretrained(("CodeFuse-13B"), device_map="auto").half().eval()

input_ids = tokenizer.encode("# language: Python\ndef quick_sort(array):\n", return_tensors="pt").to("cuda")
output_ids = model.generate(input_ids, max_new_tokens=200)

print(tokenizer.decode(output_ids[0]))

✨ Features

CodeFuse-13B is a 13 billion parameter code generation model trained on the GPT-NeoX framework, capable of handling code sequences of up to 4096 characters. This model was pretrained on a dataset consisting of 1000B token code, Chinese, and English data, covering over 40 programming languages. To further enhance the effectiveness and quality of the generated code, the model was fine-tuned on the CodeFuse-Evol-instruction-66k dataset, enabling it to produce more accurate, efficient, and compliant code. Pass@1 achieved 37.1% on the HumanEval evaluation set(BeamSearch strategy, BeamSize=3).

📦 Installation

• Python 3.8 or above.
• PyTorch 1.12 or above, with a recommendation for 2.0 or above.
• Transformers 4.24.0 or above.
• It is advisable to use CUDA 11.4 or above (for GPU users and flash-attention users, this option should be considered).

📚 Documentation

Code Community

Homepage: 🏡 https://github.com/codefuse-ai (Please give us your support with a Star🌟 + Fork🚀 + Watch👀)

• If you wish to fine-tune the model yourself, you can visit ✨MFTCoder✨✨

• If you wish to deploy the model yourself, you can visit ✨FasterTransformer4CodeFuse✨✨

• If you wish to see a demo of the model, you can visit ✨CodeFuse Demo✨✨

MD5

We notice that the file may be corrupted during transfer process. Please check MD5 value before use.

📄 License

other