Granite 8b Code Instruct 4k

🚀 Granite-8B-Code-Instruct-4K

Granite-8B-Code-Instruct-4K is an 8B parameter model fine-tuned from Granite-8B-Code-Base-4K. It's optimized on a combination of permissively licensed instruction data to enhance instruction-following capabilities, including logical reasoning and problem-solving skills.

🚀 Quick Start

The model is designed to respond to coding-related instructions and can be used to build coding assistants. Here is a simple example of how to use the Granite-8B-Code-Instruct-4K model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # or "cpu"
model_path = "ibm-granite/granite-8b-code-instruct-4k"
tokenizer = AutoTokenizer.from_pretrained(model_path)
# drop device_map if running on CPU
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()
# change input text as desired
chat = [
    { "role": "user", "content": "Write a code to find the maximum value in a list of numbers." },
]
chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
# tokenize the text
input_tokens = tokenizer(chat, return_tensors="pt")
# transfer tokenized inputs to the device
for i in input_tokens:
    input_tokens[i] = input_tokens[i].to(device)
# generate output tokens
output = model.generate(**input_tokens, max_new_tokens=100)
# decode output tokens into text
output = tokenizer.batch_decode(output)
# loop over the batch to print, in this example the batch size is 1
for i in output:
    print(i)

✨ Features

• Enhanced Instruction Following: Fine-tuned to better follow coding instructions, improving logical reasoning and problem-solving abilities.
• Multilingual Support: Trained on data from 92 programming languages, offering broad language coverage.

📦 Installation

No specific installation steps were provided in the original README.

📚 Documentation

Model Summary

• Developers: IBM Research
• GitHub Repository: ibm-granite/granite-code-models
• Paper: Granite Code Models: A Family of Open Foundation Models for Code Intelligence
• Release Date: May 6th, 2024
• License: Apache 2.0

Training Data

Granite Code Instruct models are trained on the following types of data:

• Code Commits Datasets: Sourced from the CommitPackFT dataset, considering data for 92 programming languages.
• Math Datasets: Using MathInstruct and MetaMathQA, with some filtering due to license issues.
• Code Instruction Datasets: Including Glaive-Code-Assistant-v3, Glaive-Function-Calling-v2, NL2SQL11 and synthetic API calling datasets.
• Language Instruction Datasets: Incorporating HelpSteer, an open license-filtered version of Platypus, and hardcoded prompts.

Infrastructure

The Granite Code models are trained using two of IBM's supercomputing clusters, Vela and Blue Vela, equipped with NVIDIA A100 and H100 GPUs respectively, providing a scalable and efficient training infrastructure.

Ethical Considerations and Limitations

The model's performance may be limited with out-of-domain programming languages. It's beneficial to provide few-shot examples. Developers should perform safety testing and target-specific tuning before deploying on critical applications. The model also inherits ethical considerations and limitations from its base model. For more information, refer to the Granite-8B-Code-Base-4K model card.

🔧 Technical Details

Model Index

📄 License

The model is licensed under the Apache 2.0 license.