๐ davidkim205/Ko-Llama-3-8B-Instruct
Ko-Llama-3-8B-Instruct is one of the models under research to enhance the performance of Korean language models. This model creates a dataset using the REJECTION SAMPLING technique and then undergoes training via Supervised Fine Tuning.
โจ Features
- Developed to improve Korean language model performance.
- Utilizes REJECTION SAMPLING for dataset creation and Supervised Fine Tuning for training.
๐ฆ Installation
If the following undefined symbol error occurs, install torch as follows:
...
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
/home/david/anaconda3/envs/spaces/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi
pip install torch==2.2.0
pip install flash-attn==2.5.9.post1
๐ป Usage Examples
Basic Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "davidkim205/Ko-Llama-3-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
while True:
prompt = input('>')
messages = [
{"role": "system", "content": "๋น์ ์ ๊ตฌ์ฒด์ ์ผ๋ก ๋ต๋ณํ๋ ์ฑ๋ด์
๋๋ค."},
{"role": "user", "content": prompt},
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = model.generate(
input_ids,
max_new_tokens=1024,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
Example Output
์ฌ๊ณผ์ ์๋ฏธ๋ฅผ ์ค๋ช
ํ์์ค
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:128009 for open - end generation.
์ฌ๊ณผ๋ ์ผ๋ฐ์ ์ผ๋ก ๋ง๊ณผ ์์๊ฐ ์๋ ๊ณผ์ผ๋ก ์๋ ค์ ธ ์์ต๋๋ค. ์ฌ๊ณผ๋ ์ ์ ํ ์ํ์์ ์ฃผ๋ก ๋จน๊ฑฐ๋, ์๊ฑฐํธ๋ ์ค๋ฌด๋ ๋ฑ์ ์๋ฃ์ ํผํฉํ์ฌ ์ญ์ทจ๋๊ธฐ๋ ํฉ๋๋ค. ๋ํ, ์ฌ๊ณผ๋ ๋ค์ํ ์ข
๋ฅ๊ฐ ์์ผ๋ฉฐ, ๊ฐ๊ฐ์ ์ข
๋ฅ๋ ๋ค๋ฅธ ์์๊ณผ ๋ง์ ๊ฐ์ง๊ณ ์์ต๋๋ค.
์ฌ๊ณผ๋ ๊ณผ์ผ์ด์ง๋ง, ์ข
์ข
๋ค๋ฅธ ์๋ฏธ๋ก๋ ์ฌ์ฉ๋ฉ๋๋ค. ์๋ฅผ ๋ค์ด, "์ฌ๊ณผ"๋ผ๋ ๋จ์ด๋ ์ด๋ค ๊ฒ์ด ์๋ชป๋๊ฑฐ๋ ๋ถ์กฑํ ๊ฒ์ ์์ฌํ๋ ์ํฉ์์ ์ฌ์ฉ๋ ์๋ ์์ต๋๋ค. ์๋ฅผ ๋ค์ด, "์ฌ๊ณผ"๋ฅผ ์ฃผ๋ ๊ฒ์ ์๋ชป๋ ํ๋์ด๋ ๋ถ์กฑํ ์ฌ๊ณ ๋ก ์ธํ ์ฌ๊ณผ๋ฅผ ์๋ฏธํ ์ ์์ต๋๋ค.
๋ํ, "์ฌ๊ณผ"๋ ์ด๋ค ์ํฉ์์ ๋ค๋ฅธ ์ฌ๋์๊ฒ์ ์ฌ๊ณผ๋ฅผ ๋ฐ๋ ๊ฒ์ ์๋ฏธํ๊ธฐ๋ ํฉ๋๋ค. ์๋ฅผ ๋ค์ด, "์ฌ๊ณผ"๋ฅผ ๊ตฌํ์ง ์์ผ๋ฉด ์ด๋ค ์ํฉ์์ ๋ค๋ฅธ ์ฌ๋์๊ฒ์ ์ฌ๊ณผ๋ฅผ ๋ฐ์ง ๋ชปํ ์๋ ์์ต๋๋ค.
๋ฐ๋ผ์, "์ฌ๊ณผ"๋ ๋ค์ํ ์๋ฏธ๋ก ์ฌ์ฉ๋๋ ๋จ์ด์ด๋ฉฐ, ๋งฅ๋ฝ์ ๋ฐ๋ผ ๋ค๋ฅธ ์๋ฏธ๋ฅผ ๊ฐ์ง ์ ์์ต๋๋ค.
๐ Documentation
Model Details
Property |
Details |
Model Developers |
davidkim (changyeon kim) |
Repository |
- |
Base Model |
meta-llama/Meta-Llama-3-8B-Instruct |
SFT Dataset |
sft_rs_140k |
Benchmark
kollm_evaluation
https://github.com/davidkim205/kollm_evaluation
task |
acc |
average |
0.47 |
kobest |
0.54 |
kobest_boolq |
0.57 |
kobest_copa |
0.62 |
kobest_hellaswag |
0.42 |
kobest_sentineg |
0.57 |
kobest_wic |
0.49 |
ko_truthfulqa |
0.29 |
ko_mmlu |
0.34 |
ko_hellaswag |
0.36 |
ko_common_gen |
0.76 |
ko_arc_easy |
0.33 |
Evaluation of KEval
KEval is an evaluation model that learned the prompt and dataset used in the benchmark for evaluating Korean language models among various methods of evaluating models with ChatGPT to compensate for the shortcomings of the existing lm-evaluation-harness.
https://huggingface.co/davidkim205/keval-7b
keval |
average |
kullm |
logickor |
wandb |
openai/gpt-4 |
6.79 |
4.66 |
8.51 |
7.21 |
openai/gpt-3.5-turbo |
6.25 |
4.48 |
7.29 |
6.99 |
davidkim205/Ko-Llama-3-8B-Instruct |
5.59 |
4.24 |
6.46 |
6.06 |
Evaluation of ChatGPT
chatgpt |
average |
kullm |
logickor |
wandb |
openai/gpt-4 |
7.30 |
4.57 |
8.76 |
8.57 |
openai/gpt-3.5-turbo |
6.53 |
4.26 |
7.5 |
7.82 |
davidkim205/Ko-Llama-3-8B-Instruct |
5.45 |
4.22 |
6.49 |
5.64 |
๐ License
This project is under the llama3 license.