Tinyllama 1.1B Intermediate 1.5T PTBR Instruct V3 8k

🚀 Finetuned TinyLlama-1.1B for Portuguese Text Generation

This is a finetuned version of PY007/TinyLlama-1.1B-intermediate-step-715k-1.5T on a Portuguese instruct dataset using axolotl. It aims to generate text based on given instructions, providing useful responses in Portuguese.

🚀 Quick Start

The model has different versions with varying context lengths. Versions v0, v1, and v2 were finetuned for the default 2048 context length. For version v3, the existing v2 model was further finetuned on an 8k context length dataset.

✨ Features

• Context Length Adaptability: The model supports different context lengths, including the default 2048 and an extended 8k in version v3.
• Basic RAG/QA: It performs well for basic Retrieval Augmented Generation (RAG) and question answering on retrieved content.

💻 Usage Examples

Basic Usage

# The prompt format for interacting with the model
instruction = "Your instruction here"
prompt = f"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:\n"
# Then use this prompt to get a response from the model

📄 License

This project is licensed under the MIT License.