Tinyllama 1.1B Chat V0.4

🚀 TinyLlama-1.1B

The TinyLlama project focuses on pretraining a 1.1B Llama model with 3 trillion tokens. Through appropriate optimization, this can be accomplished in "just" 90 days using 16 A100 - 40G GPUs 🚀🚀. The training commenced on September 1st, 2023.

TinyLlama adopts the same architecture and tokenizer as Llama 2. This enables it to be easily integrated into numerous open - source projects based on Llama. Moreover, with only 1.1B parameters, TinyLlama is highly compact, making it suitable for various applications with limited computation and memory requirements.

🚀 Quick Start

This Model

This is a chat model fine - tuned on TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T. The dataset used is OpenAssistant/oasst_top1_2023-08-25, following the chatml format.

How to use

You need transformers >= 4.31. For more information, check the TinyLlama GitHub page.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer
import transformers 
import torch
model = "PY007/TinyLlama-1.1B-Chat-v0.4"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

CHAT_EOS_TOKEN_ID = 32002

prompt = "How to get in a good university?"
formatted_prompt = (
    f"<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n"
)


sequences = pipeline(
    formatted_prompt,
    do_sample=True,
    top_k=50,
    top_p = 0.9,
    num_return_sequences=1,
    repetition_penalty=1.1,
    max_new_tokens=1024,
    eos_token_id=CHAT_EOS_TOKEN_ID,
)

for seq in sequences:
    print(f"Result: {seq['generated_text']}")

📄 License

This project is under the Apache - 2.0 license.

📦 Training Datasets