đ raincandy-u/TinyChat-1776K
A tiny Language Model (LM) trained from scratch on the TinyChat dataset, aiming to achieve natural responses with the smallest possible model.
This model is trained using a dataset of 3 - year - old children level English conversations. It's important to note that it has no world knowledge, so you shouldn't ask it any intellectual questions.
đ Quick Start
This section provides an overview of the model and its limitations. Remember, since it lacks world knowledge, avoid asking intellectual questions.
⨠Features
- Trained from scratch on the TinyChat dataset.
- Aims to achieve natural responses with a minimal model.
- Uses a dataset of 3 - year - old children level English conversations.
đ Documentation
đ§ Model Spec
The following code shows the configuration of the model:
config = AutoConfig.for_model(
model_type="llama",
hidden_size=192,
intermediate_size=640,
num_attention_heads=16,
num_hidden_layers=3,
num_key_value_heads=4,
tie_word_embeddings=True,
vocab_size=2048,
max_position_embeddings=256
)
đģ Usage Examples
đ Basic Usage
Here is a template for using the model:
print("<A>Hi, Tom. How are you? <end>")
print("<B>I'm fine, thank you. And you? <end>")
print("<A>Fine. What's your favorite color? <end>")
print("<B>My favorite color is black. <end>")
print("<A>Do you like cats? <end>")
print("<B>")
đ Advanced Usage
The example output for the above template is:
print("Yes, I do. I like it too. They are good for me.")
âī¸ Generation Param
When generating text, you can use the following parameters:
top_k=40,
top_p=0.8,
temperature=1
đ License
This project is licensed under the Apache - 2.0 license.
đ Information Table
Property |
Details |
Model Type |
Llama - based custom model |
Training Data |
raincandy - u/TinyChat |
â ī¸ Important Note
â ī¸ Important Note
The model has no world knowledge, so you should not ask it any intellectual questions.
đĄ Usage Tip
đĄ Usage Tip
Use the provided template and generation parameters to get the best results from the model.