Starcoder2-3b-GGUF Open-source Code Generation Model - Freely Deploy and Write Code Effortlessly

Starcoder2 3b GGUF

Developed by tensorblock

StarCoder2-3B is an open-source model with 3 billion parameters focused on code generation, developed by the BigCode team.

Large Language Model

Transformers

OtherOpen Source License:Openrail #Code Generation #Programming Assistance #Low-Resource Quantization

Downloads 103

Release Time : 11/8/2024

Model Overview

StarCoder2-3B is a text generation model, particularly proficient in code generation tasks. It is trained on the bigcode/the-stack-v2-train dataset and supports code generation and completion in multiple programming languages.

Model Features

Efficient Code Generation

Performs excellently on multiple code generation evaluation datasets, such as achieving a pass@1 of 31.7 on HumanEval.

Multiple Quantization Versions

Provides a total of 12 quantization versions from Q2_K to Q8_0 to meet the inference requirements under different hardware conditions.

Open-Source License

Adopts the bigcode-openrail-m license, allowing commercial and research use.

Model Capabilities

Code Generation

Code Completion

Programming Problem Solving

Use Cases

Software Development

Automated Code Generation

Generate runnable code snippets based on natural language descriptions

Achieves a pass@1 of 31.7 on the HumanEval dataset

Code Completion

Provide intelligent code completion suggestions in the IDE

Achieves an edit similarity of 71.19 on RepoBench-v1.1

Programming Education

Programming Problem Solving

Help students understand and solve programming problems

Achieves an accuracy of 27.7 on the GSM8K(PAL) dataset

🚀 bigcode/starcoder2-3b - GGUF

This repository provides GGUF format model files for bigcode/starcoder2-3b. These files are quantized using machines from TensorBlock, and are compatible with llama.cpp as of commit b4011.

🚀 Quick Start

The following sections will introduce the basic information, related projects, model file specifications, and downloading instructions of this model.

✨ Features

Model Compatibility: The GGUF format model files are compatible with llama.cpp, facilitating seamless integration into relevant projects.
Quantization: Quantized using TensorBlock's machines, optimizing the model for various applications.

📦 Installation

Command line

First, install the Huggingface Client:

pip install -U "huggingface_hub[cli]"

Then, download the individual model file to a local directory:

huggingface-cli download tensorblock/starcoder2-3b-GGUF --include "starcoder2-3b-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you want to download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/starcoder2-3b-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'

📚 Documentation

Our projects

Project Name	Description	Link
Forge	An OpenAI-compatible multi-provider routing layer.	Try it now!
Awesome MCP Servers	A comprehensive collection of Model Context Protocol (MCP) servers.	See what we built
TensorBlock Studio	A lightweight, open, and extensible multi-LLM interaction studio.	See what we built

Model file specification

Filename	Quant type	File Size	Description
starcoder2-3b-Q2_K.gguf	Q2_K	1.139 GB	smallest, significant quality loss - not recommended for most purposes
starcoder2-3b-Q3_K_S.gguf	Q3_K_S	1.273 GB	very small, high quality loss
starcoder2-3b-Q3_K_M.gguf	Q3_K_M	1.455 GB	very small, high quality loss
starcoder2-3b-Q3_K_L.gguf	Q3_K_L	1.618 GB	small, substantial quality loss
starcoder2-3b-Q4_0.gguf	Q4_0	1.629 GB	legacy; small, very high quality loss - prefer using Q3_K_M
starcoder2-3b-Q4_K_S.gguf	Q4_K_S	1.642 GB	small, greater quality loss
starcoder2-3b-Q4_K_M.gguf	Q4_K_M	1.758 GB	medium, balanced quality - recommended
starcoder2-3b-Q5_0.gguf	Q5_0	1.964 GB	legacy; medium, balanced quality - prefer using Q4_K_M
starcoder2-3b-Q5_K_S.gguf	Q5_K_S	1.964 GB	large, low quality loss - recommended
starcoder2-3b-Q5_K_M.gguf	Q5_K_M	2.031 GB	large, very low quality loss - recommended
starcoder2-3b-Q6_K.gguf	Q6_K	2.320 GB	very large, extremely low quality loss
starcoder2-3b-Q8_0.gguf	Q8_0	3.003 GB	very large, extremely low quality loss - not recommended

📄 License

This project is licensed under the bigcode-openrail-m license.

Property	Details
Pipeline Tag	text-generation
Inference	true
Widget Example	- text: 'def print_hello_world():', example_title: Hello world, group: Python
Datasets	bigcode/the-stack-v2-train
License	bigcode-openrail-m
Library Name	transformers
Tags	code, TensorBlock, GGUF
Base Model	bigcode/starcoder2-3b

Model Index

Model Name	Task Type	Dataset Name	Dataset Type	Metric Type	Metric Value
starcoder2-3b	text-generation	CruxEval-I	cruxeval-i	pass@1	32.7
starcoder2-3b	text-generation	DS-1000	ds-1000	pass@1	25.0
starcoder2-3b	text-generation	GSM8K (PAL)	gsm8k-pal	accuracy	27.7
starcoder2-3b	text-generation	HumanEval+	humanevalplus	pass@1	27.4
starcoder2-3b	text-generation	HumanEval	humaneval	pass@1	31.7
starcoder2-3b	text-generation	RepoBench-v1.1	repobench-v1.1	edit-smiliarity	71.19

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご