Llama 3 Alpha Ko 8B Instruct

🚀 Alpha-Instruct

Our latest language model excels in Korean and English, leveraging Evolutionary Model Merging for complex language tasks and logical reasoning.

We are thrilled to introduce Alpha-Instruct, our latest language model, which demonstrates exceptional capabilities in both Korean and English. Alpha-Instruct is developed using the Evolutionary Model Merging technique, enabling it to excel in complex language tasks and logical reasoning.

A key aspect of Alpha-Instruct's development is our community-based approach. We draw inspiration and ideas from various communities, shaping our datasets, methodologies, and the model itself. In return, we are committed to sharing our insights with the community, providing detailed information on the data, methods, and models used in Alpha-Instruct's creation.

Alpha-Instruct has achieved outstanding performance on the LogicKor, scoring an impressive 6.62. Remarkably, this performance rivals that of 70B models, showcasing the efficiency and power of our 8B model. This achievement highlights Alpha-Instruct's advanced computational and reasoning skills, making it a leading choice for diverse and demanding language tasks.

For more information and technical details about Alpha-Instruct, stay tuned to our updates and visit our website (Soon).

✨ Features

• Demonstrates exceptional capabilities in both Korean and English.
• Developed using the Evolutionary Model Merging technique, excelling in complex language tasks and logical reasoning.
• Adopts a community - based approach in development and shares insights with the community.
• Achieved an outstanding score of 6.62 on LogicKor, comparable to 70B models.

📚 Documentation

Overview

Alpha-Instruct is our latest language model, developed using 'Evolutionary Model Merging' technique. This method employs a 1:1 ratio of task - specific datasets from KoBEST and Haerae, resulting in a model named 'Alpha - Ko - 8B - Evo'. The following models were used for merging:

• [Meta - Llama - 3 - 8B](https://huggingface.co/meta - llama/Meta - Llama - 3 - 8B) (Base)
• [Meta - Llama - 3 - 8B - Instruct](https://huggingface.co/meta - llama/Meta - Llama - 3 - 8B - Instruct) (Instruct)
• [Llama - 3 - Open - Ko - 8B](beomi/Llama - 3 - Open - Ko - 8B) (Continual Pretrained)

To refine and enhance Alpha-Instruct, we utilized carefully curated high - quality datasets aimed at 'healing' the model's output, significantly boosting its human preference scores. We use ORPO specifically for this "healing" (RLHF) phase. The datasets* used include:

• [Korean - Human - Judgements](https://huggingface.co/datasets/HAERAE - HUB/Korean - Human - Judgements)
• [Orca - Math](https://huggingface.co/datasets/kuotient/orca - math - word - problems - 193k - Korean)
• [dpo - mix - 7k](https://huggingface.co/datasets/argilla/dpo - mix - 7k)

*Some of these datasets were partially used and translated for training, and we ensured there was no contamination during the evaluation process.

This approach effectively balances human preferences with the model's capabilities, making Alpha-Instruct well - suited for real - life scenarios where user satisfaction and performance are equally important.

Benchmark Results

Results in LogicKor* are as follows:

*Self report(Default settings with 'alpha' template, mean of 3).

Results in KoBEST(acc, num_shot = 5) are as follows:

*For reference, 'merged' models are chosen.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "allganize/Llama-3-Alpha-Ko-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "system", "content": "당신은 인공지능 어시스턴트입니다. 묻는 말에 친절하고 정확하게 답변하세요."},
    {"role": "user", "content": "피보나치 수열이 뭐야? 그리고 피보나치 수열에 대해 파이썬 코드를 짜줘볼래?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    eos_token_id=terminators,
    do_sample=False,
    repetition_penalty=1.05,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

📄 License

The use of this model is governed by the META LLAMA 3 COMMUNITY LICENSE AGREEMENT

📚 Citation

If you use this model in your research, please cite it as follows:

@misc{alpha-instruct,
  author       = {Ji soo Kim},
  title        = {Alpha-Instruct: Allganize Bilingual Model},
  year         = {2024},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  url          = {https://huggingface.co/allganize/Llama-3-Alpha-Ko-8B-Instruct},
}

👥 Contact

Correspondence to

• Ji soo Kim (jisoo.kim@allganize.ai)
• Contributors
  • Sangmin Jeon (sangmin.jeon@allganize.ai)
  • Seungwoo Ryu (seungwoo.ryu@allganize.ai)

Special Thanks

• @beomi for providing us with a great model!