Tinyllama 1.1B Chat V0.1

🚀 TinyLlama-1.1B

The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens, offering a compact solution for various applications with limited computation and memory requirements.

🚀 Quick Start

The TinyLlama project is on a mission to pretrain a 1.1B Llama model on 3 trillion tokens. With appropriate optimization, this can be accomplished in "just" 90 days using 16 A100 - 40G GPUs 🚀🚀. The training kicked off on 2023 - 09 - 01.

TinyLlama adopts the exact same architecture and tokenizer as Llama 2. This enables it to be easily integrated into many open - source projects based on Llama. Moreover, with only 1.1B parameters, TinyLlama is highly compact, making it suitable for a wide range of applications that demand restricted computation and memory usage.

📚 Documentation

This Model

This is a chat model fine - tuned on PY007/TinyLlama-1.1B-intermediate-step-240k-503b. The dataset used is [openassistant - guananco](https://huggingface.co/datasets/timdettmers/openassistant - guanaco).

How to use

You will need transformers>=4.31. For more information, do check the TinyLlama GitHub page.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer
import transformers 
import torch
model = "PY007/TinyLlama-1.1B-Chat-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

prompt = "What are the values in open source projects?"
formatted_prompt = (
    f"### Human: {prompt}### Assistant:"
)


sequences = pipeline(
    formatted_prompt,
    do_sample=True,
    top_k=50,
    top_p = 0.7,
    num_return_sequences=1,
    repetition_penalty=1.1,
    max_new_tokens=500,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

📄 License

This project is licensed under the Apache - 2.0 license.

📦 Dataset