🚀 OpenChat: Less is More for Open-source Models
OpenChat is a series of open - source language models. These models are fine - tuned on a diverse and high - quality multi - round conversation dataset. By filtering only about 6K GPT - 4 conversations from approximately 90K ShareGPT conversations, OpenChat aims to achieve high performance with limited data.
✨ Features
Generic models
- OpenChat: Based on LLaMA - 13B (2048 context length)
- Achieves 🚀 105.7% of ChatGPT score on Vicuna GPT - 4 evaluation.
- Has a 🔥 80.9% win - rate on AlpacaEval.
- 🤗 Only used 6K data for finetuning!!!
- OpenChat - 8192: Based on LLaMA - 13B (extended to 8192 context length)
- Gets 106.6% of ChatGPT score on Vicuna GPT - 4 evaluation.
- Has a 79.5% win - rate on AlpacaEval.
Code models
- OpenCoderPlus: Based on StarCoderPlus (native 8192 context length)
- Achieves 102.5% of ChatGPT score on Vicuna GPT - 4 evaluation.
- Has a 78.7% win - rate on AlpacaEval.
⚠️ Important Note
Please load the pretrained models using bfloat16.
📦 Installation
No installation steps are provided in the original document, so this section is skipped.
💻 Usage Examples
Basic Usage
The conversation template involves concatenating tokens.
Besides base model vocabulary, an end - of - turn token <|end_of_turn|>
is added, with id eot_token_id
.
[bos_token_id] + tokenize("Human: ") + tokenize(user_question) + [eot_token_id] + tokenize("Assistant: ")
tokenize("User:") + tokenize(user_question) + [eot_token_id] + tokenize("Assistant:")
💡 Usage Tip
In BPE, tokenize(A) + tokenize(B)
does not always equal to tokenize(A + B)
.
Advanced Usage
Following is the code for generating the conversation templates:
@dataclass
class ModelConfig:
system: Optional[str]
role_prefix: dict
ai_role: str
eot_token: str
bos_token: Optional[str] = None
def generate_conversation_template(self, tokenize_fn, tokenize_special_fn, message_list):
tokens = []
masks = []
if self.bos_token:
t = tokenize_special_fn(self.bos_token)
tokens.append(t)
masks.append(False)
if self.system:
t = tokenize_fn(self.system) + [tokenize_special_fn(self.eot_token)]
tokens.extend(t)
masks.extend([False] * len(t))
for idx, message in enumerate(message_list):
t = tokenize_fn(self.role_prefix[message["from"]])
tokens.extend(t)
masks.extend([False] * len(t))
if "value" in message:
t = tokenize_fn(message["value"]) + [tokenize_special_fn(self.eot_token)]
tokens.extend(t)
masks.extend([message["from"] == self.ai_role] * len(t))
else:
assert idx == len(message_list) - 1, "Empty message for completion must be on the last."
return tokens, masks
MODEL_CONFIG_MAP = {
"openchat": ModelConfig(
system=None,
role_prefix={
"human": "Human: ",
"gpt": "Assistant: "
},
ai_role="gpt",
eot_token="<|end_of_turn|>",
bos_token="<s>",
),
"opencoder": ModelConfig(
system=None,
role_prefix={
"human": "User:",
"gpt": "Assistant:"
},
ai_role="gpt",
eot_token="<|end_of_turn|>",
bos_token=None,
)
}
📚 Documentation
Code and Inference Server
We provide the full source code, including an inference server compatible with the "ChatCompletions" API, in the OpenChat GitHub repository.
Web UI
OpenChat also includes a web UI for a better user experience. See the GitHub repository for instructions.
📄 License
Our weight license is subject to their corresponding base model. For example, OpenChat and OpenChat - 8192 are the same as the model License of LLaMA for non - commercial use only, while OpenCoderPlus is under the License of StarCoder. Furthermore, we should follow the [Privacy Practices](https://chrome.google.com/webstore/detail/sharegpt - share - your - chatg/daiacboceoaocpibfodeljbdfacokfjb) of ShareGPT. The code released on GitHub is under Apache License 2.0.
📚 Citation
@software{openllms23,
title = {{OpenLLMs: Less is More for Open-source Models}},
author = {Wang, Guan and Cheng, Sijie and Yu, Qiying and Liu, Changling},
doi = {10.5281/zenodo.8105775},
url = {https://github.com/imoneoi/openchat},
version = {pre-release},
year = {2023},
month = {7},
}