Starcoder2 3b Instruct

🚀 starcoder2-instruct

We've fine-tuned starcoder2-3b with additional high-quality code tokens, achieving 65.9 pass@1 on HumanEval-Python. This model uses the Alpaca instruction format.

🚀 Quick Start

We've fine-tuned starcoder2-3b with an additional 0.7 billion high-quality, code-related tokens for 3 epochs. We used DeepSpeed ZeRO 3 and Flash Attention 2 to accelerate the training process. It achieves 65.9 pass@1 on HumanEval-Python. This model operates using the Alpaca instruction format (excluding the system prompt).

✨ Features

• Fine-tuned with additional high-quality code tokens.
• Accelerated training using DeepSpeed ZeRO 3 and Flash Attention 2.
• Achieves 65.9 pass@1 on HumanEval-Python.
• Operates using the Alpaca instruction format.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
PROMPT = """### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
tokenizer = AutoTokenizer.from_pretrained("TechxGenus/starcoder2-3b-instruct")
model = AutoModelForCausalLM.from_pretrained(
    "TechxGenus/starcoder2-3b-instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=2048)
print(tokenizer.decode(outputs[0]))

Advanced Usage

from transformers import pipeline
import torch
PROMPT = """### Instruction
{instruction}
### Response
"""
instruction = <Your code instruction here>
prompt = PROMPT.format(instruction=instruction)
generator = pipeline(
    model="TechxGenus/starcoder2-3b-instruct",
    task="text-generation",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
result = generator(prompt, max_length=2048)
print(result[0]["generated_text"])

📄 License

This model is licensed under the bigcode-openrail-m license.

⚠️ Important Note

Model may sometimes make errors, produce misleading contents, or struggle to manage tasks that are not related to coding. It has undergone very limited testing. Additional safety testing should be performed before any real-world deployments.