Tinyllama 1.1B Alpaca Chat V1.5 GGUF

🚀 tog/TinyLlama-1.1B-alpaca-chat-v1.5-GGUF

This project provides quantized GGUF model files for TinyLlama-1.1B-alpaca-chat-v1.5 from tog. These files are designed to optimize the performance and resource usage of the model.

📦 Model Information

📋 Quantized Model Files

📚 Original Model Card

This Model

This is a chat model fine - tuned on top of PY007/TinyLlama-1.1B-intermediate-step-715k-1.5T. The dataset used for fine - tuning is tatsu-lab/stanford_alpaca.

The model follows the instruction - response format:

### Instruction:
{instruction}

### Response:

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer
import transformers
import torch

model = "tog/TinyLlama-1.1B-alpaca-chat-v1.5"
tokenizer = AutoTokenizer.from_pretrained(model)

pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto")

sequences = pipeline(
    '###Instruction:\nWhat is a large language model? Be concise.\n\n### Response:\n',
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=200)

for seq in sequences:
    print(f"{seq['generated_text']}")

When you run the above code, you should get an output similar to this:

Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Result: ###Instruction:
What is a large language model? Be concise.

### Response:
A large language model is a type of natural language understanding model that can learn to accurately recognize and interpret text data by understanding the context of words. Languages used for text understanding are typically trained on a corpus of text data.

📄 License

This project is licensed under the apache - 2.0 license.