đ Model Card for StarChat Alpha
StarChat is a series of language models fine - tuned from StarCoder to serve as useful coding assistants. StarChat Alpha is the first in this series. As an alpha release, it's only for educational or research purposes. It hasn't been aligned with human preferences using techniques like RLHF, so it may generate problematic content, especially when prompted to do so.
đ Quick Start
Use the following code to start using the model. Here's how you can run the model with the pipeline()
function from đ¤ Transformers:
import torch
from transformers import pipeline
pipe = pipeline("text-generation", model="HuggingFaceH4/starchat-alpha", torch_dtype=torch.bfloat16, device_map="auto")
prompt_template = "<|system|>\n<|end|>\n<|user|>\n{query}<|end|>\n<|assistant|>"
prompt = prompt_template.format(query="How do I sort a list in Python?")
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.2, top_k=50, top_p=0.95, eos_token_id=49155)
⨠Features
StarChat Alpha is a 16B parameter GPT - like model fine - tuned on a blend of the oasst1
and databricks-dolly-15k
datasets. It can be used to explore the programming capabilities of open - source language models for educational and research purposes.
đ Documentation
Model Details
Model Description
Property |
Details |
Model Type |
A 16B parameter GPT - like model fine - tuned on a blend of the oasst1 and databricks-dolly-15k datasets. |
Language(s) (NLP) |
English |
License |
BigCode Open RAIL - M v1 |
Finetuned from model |
bigcode/starcoderbase |
Model Sources
- Repository: https://github.com/bigcode-project/starcoder
- Demo: https://huggingface.co/spaces/HuggingFaceH4/starchat-playground
Uses
StarChat Alpha is designed for educational and/or research purposes. It can be used to test the programming capabilities of open - source language models.
Bias, Risks, and Limitations
â ī¸ Important Note
StarChat Alpha has not been aligned to human preferences with techniques like RLHF or deployed with in - the - loop filtering of responses like ChatGPT. So, it can produce problematic outputs, especially when prompted to do so. Models trained mainly on code data will also have a more skewed demographic bias similar to the demographics of the GitHub community. For more on this, see the StarCoder dataset which is derived from The Stack.
Since the base model was pretrained on a large code corpus, it may generate code snippets that are syntactically correct but semantically wrong. For example, it may generate code that doesn't compile or produces incorrect results. It may also generate code vulnerable to security exploits. We've noticed that the model also tends to generate false URLs, which should be carefully checked before clicking.
StarChat Alpha was fine - tuned from the base model StarCoder Base. Please refer to its model card's Limitations Section for relevant information. In particular, the model was evaluated on some categories of gender biases, propensity for toxicity, and risk of suggesting code completions with known security flaws. These evaluations are reported in its [technical report](https://drive.google.com/file/d/1cN - b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view).
đ License
The model is licensed under BigCode Open RAIL - M v1.
đ Citation
BibTeX:
@article{Tunstall2023starchat-alpha,
author = {Tunstall, Lewis and Lambert, Nathan and Rajani, Nazneen and Beeching, Edward and Le Scao, Teven and von Werra, Leandro and Han, Sheon and Schmid, Philipp and Rush, Alexander},
title = {Creating a Coding Assistant with StarCoder},
journal = {Hugging Face Blog},
year = {2023},
note = {https://huggingface.co/blog/starchat},
}