Starchat Alpha

🚀 Model Card for StarChat Alpha

StarChat is a series of language models fine - tuned from StarCoder to serve as useful coding assistants. StarChat Alpha is the first in this series. As an alpha release, it's only for educational or research purposes. It hasn't been aligned with human preferences using techniques like RLHF, so it may generate problematic content, especially when prompted to do so.

🚀 Quick Start

Use the following code to start using the model. Here's how you can run the model with the pipeline() function from 🤗 Transformers:

import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceH4/starchat-alpha", torch_dtype=torch.bfloat16, device_map="auto")

prompt_template = "<|system|>\n<|end|>\n<|user|>\n{query}<|end|>\n<|assistant|>"
prompt = prompt_template.format(query="How do I sort a list in Python?")
# We use a special <|end|> token with ID 49155 to denote ends of a turn
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.2, top_k=50, top_p=0.95, eos_token_id=49155)
# You can sort a list in Python by using the sort() method. Here's an example:\n\n```\nnumbers = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5]\nnumbers.sort()\nprint(numbers)\n```\n\nThis will sort the list in place and print the sorted list.

✨ Features

StarChat Alpha is a 16B parameter GPT - like model fine - tuned on a blend of the oasst1 and databricks-dolly-15k datasets. It can be used to explore the programming capabilities of open - source language models for educational and research purposes.

📚 Documentation

Model Details

Model Description

Model Sources

• Repository: https://github.com/bigcode-project/starcoder
• Demo: https://huggingface.co/spaces/HuggingFaceH4/starchat-playground

Uses

StarChat Alpha is designed for educational and/or research purposes. It can be used to test the programming capabilities of open - source language models.

Bias, Risks, and Limitations

⚠️ Important Note

StarChat Alpha has not been aligned to human preferences with techniques like RLHF or deployed with in - the - loop filtering of responses like ChatGPT. So, it can produce problematic outputs, especially when prompted to do so. Models trained mainly on code data will also have a more skewed demographic bias similar to the demographics of the GitHub community. For more on this, see the StarCoder dataset which is derived from The Stack.

Since the base model was pretrained on a large code corpus, it may generate code snippets that are syntactically correct but semantically wrong. For example, it may generate code that doesn't compile or produces incorrect results. It may also generate code vulnerable to security exploits. We've noticed that the model also tends to generate false URLs, which should be carefully checked before clicking.

StarChat Alpha was fine - tuned from the base model StarCoder Base. Please refer to its model card's Limitations Section for relevant information. In particular, the model was evaluated on some categories of gender biases, propensity for toxicity, and risk of suggesting code completions with known security flaws. These evaluations are reported in its [technical report](https://drive.google.com/file/d/1cN - b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view).

📄 License

The model is licensed under BigCode Open RAIL - M v1.

📖 Citation

BibTeX:

@article{Tunstall2023starchat-alpha,
  author = {Tunstall, Lewis and Lambert, Nathan and Rajani, Nazneen and Beeching, Edward and Le Scao, Teven and von Werra, Leandro and Han, Sheon and Schmid, Philipp and Rush, Alexander},
  title = {Creating a Coding Assistant with StarCoder},
  journal = {Hugging Face Blog},
  year = {2023},
  note = {https://huggingface.co/blog/starchat},
}