CodeBooga-34B-v0.1 Open-Source Code Generation Model - Free to Power Efficient Code Writing Tasks

Codebooga 34B V0.1

Developed by oobabooga

CodeBooga-34B-v0.1 is a large-scale code generation model created by merging Phind-CodeLlama-34B-v2 and WizardCoder-Python-34B-V1.0 using the BlockMerge Gradient script, focusing on code generation tasks.

Large Language Model

Transformers

#Code Generation #Programming Q&A #Multilingual Programming

Downloads 124

Release Time : 10/19/2023

Model Overview

This model is a 34B-parameter code generation model that combines two excellent code generation models, aiming to provide high-quality code generation capabilities.

Model Features

Model Fusion

Merges Phind-CodeLlama-34B-v2 and WizardCoder-Python-34B-V1.0 using the BlockMerge Gradient script, combining the strengths of both.

High-Performance Code Generation

In informal testing, this model outperforms the base models in Python and JavaScript code generation tasks.

Alpaca Prompt Format

Supports Alpaca-format prompts for easy integration with existing tools.

Model Capabilities

Code Generation

Programming Problem Solving

Code Completion

Use Cases

Software Development

Python Code Generation

Generates complex Python code to solve practical problems.

Outperforms base models in testing.

JavaScript Code Generation

Generates JavaScript code to solve front-end development issues.

Outperforms base models in testing.

🚀 CodeBooga-34B-v0.1

CodeBooga-34B-v0.1 is a merged model that combines the strengths of two powerful base models, offering enhanced performance in coding tasks.

🚀 Quick Start

This model is a merge of the following two models:

It was created using the BlockMerge Gradient script, the same script used to create MythoMax-L2-13b, with identical settings. The following YAML configuration was employed:

model_path1: "Phind_Phind-CodeLlama-34B-v2_safetensors"
model_path2: "WizardLM_WizardCoder-Python-34B-V1.0_safetensors"
output_model_path: "CodeBooga-34B-v0.1"
operations:
  - operation: lm_head # Single tensor
    filter: "lm_head"
    gradient_values: [0.75]
  - operation: embed_tokens # Single tensor
    filter: "embed_tokens"
    gradient_values: [0.75]
  - operation: self_attn
    filter: "self_attn"
    gradient_values: [0.75, 0.25]
  - operation: mlp
    filter: "mlp"
    gradient_values: [0.25, 0.75]
  - operation: layernorm
    filter: "layernorm"
    gradient_values: [0.5, 0.5]
  - operation: modelnorm # Single tensor
    filter: "model.norm"
    gradient_values: [0.75]

💻 Usage Examples

Basic Usage - Prompt Format

Both base models utilize the Alpaca format, so it should be used for this model as well.

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Your instruction

### Response:
Bot reply

### Instruction:
Another instruction

### Response:
Bot reply

📚 Documentation

Evaluation

(This is not very scientific, so bear with me.)

I conducted a quick experiment where I posed a set of 3 Python and 3 JavaScript questions (real - world, difficult questions with nuance) to the following models:

This model (CodeBooga-34B-v0.1)
A second variant generated by swapping model_path1 and model_path2 in the above YAML, named CodeBooga-Reversed-34B-v0.1
WizardCoder-Python-34B-V1.0
Phind-CodeLlama-34B-v2

Specifically, I used 4.250b EXL2 quantizations of each model. I then sorted the responses for each question by quality and assigned the following scores:

4th place: 0
3rd place: 1
2nd place: 2
1st place: 4

The resulting cumulative scores were:

CodeBooga-34B-v0.1: 22
WizardCoder-Python-34B-V1.0: 12
Phind-CodeLlama-34B-v2: 7
CodeBooga-Reversed-34B-v0.1: 1

CodeBooga-34B-v0.1 performed very well, while its variant performed poorly. Therefore, I uploaded the former but not the latter.

📦 Quantized versions

GGUF

TheBloke has kindly provided GGUF quantizations for llama.cpp:

https://huggingface.co/TheBloke/CodeBooga-34B-v0.1-GGUF

📄 License

The model uses the llama2 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご