🚀 instructTrans
The instructTrans model is designed for English-to-Korean translation, trained on specific datasets to enhance translation performance.
🚀 Quick Start
Loading the Model
Use the following Python code to load the model:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "nayohan/llama3-instrucTrans-enko-8b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
device_map="auto",
torch_dtype=torch.bfloat16
)
Generating Text
This model supports translation from English to Korean. To translate text, use the following Python code:
system_prompt="당신은 번역기 입니다. 영어를 한국어로 번역하세요."
sentence = "The aerospace industry is a flower in the field of technology and science."
conversation = [{'role': 'system', 'content': system_prompt},
{'role': 'user', 'content': sentence}]
inputs = tokenizer.apply_chat_template(
conversation,
tokenize=True,
add_generation_prompt=True,
return_tensors='pt'
).to("cuda")
outputs = model.generate(inputs, max_new_tokens=4096)
print(tokenizer.decode(outputs[0][len(inputs[0]):]))
✨ Features
📚 Documentation
Model Information
Property |
Details |
Library Name |
transformers |
Tags |
translation, enko, ko |
Base Model |
meta-llama/Meta-Llama-3-8B-Instruct |
Datasets |
nayohan/aihub-en-ko-translation-1.2m, nayohan/translate_corpus_313k |
Pipeline Tag |
text-generation |
Metrics |
sacrebleu |
License |
llama3 |
Evaluation
Evaluation Datasets Sources
Model Evaluation Method
- Each model is inferred based on the inference code written in the README on Hugging Face. (Common: max_new_tokens = 512)
- EEVE adds the instruction ("당신은 번역기 입니다. 영어를 한국어로 번역하세요.") to the system prompt, while KULLM3 keeps the original system prompt and adds it to the front of the user's input.
Aihub English-to-Korean Translation Dataset Evaluation
- The Aihub evaluation dataset may be included in the training datasets of the models. Please refer to it only for checking the performance by category. [Category description link]
| model | aihub-111 | aihub-124 | aihub-125 | aihub-126 | aihub-563 | aihub-71265 | aihub-71266 | aihub-71382 | average |
|:-----------------|------------:|------------:|------------:|------------:|------------:|--------------:|--------------:|--------------:|----------:|
| EEVE-10.8b-it | 6.15 | 11.81 | 5.78 | 4.99 | 6.31 | 10.99 | 9.41 | 6.44 | 7.73 |
| KULLM3 | 9.00 | 13.49 | 10.43 | 5.90 | 1.92 | 16.37 | 10.02 | 8.39 | 9.44 |
| Seagull-13B | 9.8 | 18.38 | 8.51 | 5.53 | 8.74 | 17.44 | 10.11 | 11.21 | 11.21 |
| Synatra-7B | 6.99 | 25.14 | 7.79 | 5.31 | 9.95 | 19.27 | 13.20 | 8.93 | 12.07 |
| nhndq-nllb | 24.09 | 48.71 | 22.89 | 13.98 | 18.71 | 30.18 | 32.49 | 18.62 | 26.20 |
| our-tech | 20.19 | 37.48 | 18.50 | 12.45 | 16.96 | 13.92 | 43.54 | 9.62 | 21.58 |
| our-general | 24.72 | 45.22 | 21.61 | 18.97 | 17.23 | 30.00 | 32.08 | 13.55 | 25.42 |
| our-sharegpt | 12.42 | 19.23 | 10.91 | 9.18 | 14.30 | 26.43 | 12.62 | 15.57 | 15.08 |
| our-instrucTrans | 24.89 | 47.00 | 22.78 | 21.78 | 24.27 | 27.98 | 31.31 | 15.42 |26.92 |
FLoRes English-to-Korean Translation Dataset Evaluation
FloRes is a translation benchmark dataset publicly released by Facebook, which is constructed in parallel for English and 200 languages with few resources.
The evaluation is conducted using traintogpb/aihub-flores-koen-integrated-sparta-30k. (Single-sentence composition)

model |
flores-dev |
flores-devtest |
average |
EEVE-10.8b-it |
10.99 |
11.71 |
11.35 |
KULLM3 |
12.83 |
13.23 |
13.03 |
Seagull-13B |
11.48 |
11.99 |
11.73 |
Synatra-7B |
10.98 |
10.81 |
10.89 |
nhndq-nllb |
12.79 |
15.15 |
13.97 |
our-tech |
12.14 |
12.04 |
12.09 |
our-general |
14.93 |
14.58 |
14.75 |
our-sharegpt |
14.71 |
16.69 |
15.70 |
our-instrucTrans |
14.49 |
17.69 |
16.09 |
iwslt-2023
The iwslt-2023 dataset consists of evaluation datasets where the same English sentences are translated into Korean in both informal and formal styles. It can relatively check the tendency of the model towards formal/informal language. (Single-sentence composition)

model |
iwslt_zondae |
iwslt_banmal |
average |
EEVE-10.8b-it |
4.62 |
3.79 |
4.20 |
KULLM3 |
5.94 |
5.24 |
5.59 |
Seagull-13B |
6.14 |
4.54 |
5.34 |
Synatra-7B |
5.43 |
4.73 |
5.08 |