Llama 3 Typhoon V1.5x 8b Instruct

🚀 Llama-3-Typhoon-1.5X-8B-instruct

Llama-3-Typhoon-1.5X-8B-instruct is an 8-billion parameter instruct model tailored for Thai and English languages. It offers competitive performance with GPT-3.5-turbo and is optimized for various application use cases, retrieval-augmented generation, constrained generation, and reasoning tasks.

🚀 Quick Start

This model requires Transformers 4.38.0 or newer. You can start using it following the usage example below.

✨ Features

• Cross - lingual Transfer: Built on Typhoon 1.5 8B and Llama 3 8B Instruct, it combines the Thai understanding of Typhoon with the human alignment of Llama 3 Instruct through cross - lingual transfer.
• Optimized for Applications: Ideal for application use cases, Retrieval - Augmented Generation (RAG), constrained generation, and reasoning tasks.

📦 Installation

The model requires Transformers 4.38.0 or newer. You can install it using the following command:

pip install transformers>=4.38.0

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "scb10x/llama-3-typhoon-v1.5x-8b-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [...] # add message here

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.4,
    top_p=0.95,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

Chat Template

We use the Llama 3 chat template:

{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}

📚 Documentation

Model Description

Performance

We evaluated the model's performance in two main aspects:

Language & Knowledge Capabilities

• Assessed using multiple - choice question - answering datasets such as ThaiExam and MMLU.

Instruction Following Capabilities

• Evaluated based on beta users' feedback, focusing on:
  • Human Alignment & Reasoning: Ability to generate clear and logically structured multi - step responses. Evaluated using MT - Bench.
  • Instruction - following: Ability to follow specified constraints in the instruction. Evaluated using IFEval.

ThaiExam Results

MT - Bench Results

IFEval Results

Insight

We used model editing techniques and found that the key to generating accurate Thai answers lies in the backend (upper layers of the transformer block). So, we incorporated a high ratio of Typhoon components in these backend layers to improve the model's performance.

🔧 Technical Details

The model is an experiment on cross - lingual transfer, utilizing the task - arithmetic model editing technique. It combines the Thai understanding of Typhoon with the human alignment of Llama 3 Instruct.

📄 License

This model is released under the Llama 3 Community License.

⚠️ Important Note

This model is experimental and might not be fully evaluated for all use cases. Developers should assess risks in the context of their specific applications.

💡 Usage Tip

If you find Typhoon - 1.5X useful for your work, please cite it using:

@article{pipatanakul2023typhoon,
    title={Typhoon: Thai Large Language Models}, 
    author={Kunat Pipatanakul and Phatrasek Jirabovonvisut and Potsawee Manakul and Sittipong Sripaisarnmongkol and Ruangsak Patomwong and Pathomporn Chokchainant and Kasima Tharnpipitchai},
    year={2023},
    journal={arXiv preprint arXiv:2312.13951},
    url={https://arxiv.org/abs/2312.13951}
}

🌐 Follow Us

https://twitter.com/opentyphoon

💬 Support

https://discord.gg/us5gAYmrxw

👥 SCB 10X Typhoon Team

• Kunat Pipatanakul, Potsawee Manakul, Sittipong Sripaisarnmongkol, Natapong Nitarach, Pathomporn Chokchainant, Kasima Tharnpipitchai

📧 Contact Us

• General & Collaboration: kasima@scb10x.com, pathomporn@scb10x.com
• Technical: kunat@scb10x.com