Llama3 Instructrans Enko 8b

🚀 instructTrans

The instructTrans model is designed for English-to-Korean translation, trained on specific datasets to enhance translation performance.

🚀 Quick Start

Loading the Model

Use the following Python code to load the model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "nayohan/llama3-instrucTrans-enko-8b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
  model_name,
  device_map="auto",
  torch_dtype=torch.bfloat16
)

Generating Text

This model supports translation from English to Korean. To translate text, use the following Python code:

system_prompt="당신은 번역기 입니다. 영어를 한국어로 번역하세요."
sentence = "The aerospace industry is a flower in the field of technology and science."
conversation = [{'role': 'system', 'content': system_prompt},
                {'role': 'user', 'content': sentence}]

inputs = tokenizer.apply_chat_template(
  conversation,
  tokenize=True,
  add_generation_prompt=True,
  return_tensors='pt'
).to("cuda")

outputs = model.generate(inputs, max_new_tokens=4096) # Finetuned with length 4096
print(tokenizer.decode(outputs[0][len(inputs[0]):]))

✨ Features

• Translation Capability: The llama3-8b-instructTrans-en-ko model is trained on translation datasets (English -> Korean) based on Llama-3-8B-it to translate English instruction datasets.
• Multiple Datasets: Trained on datasets like nayohan/aihub-en-ko-translation-1.2m and nayohan/translate_corpus_313k.

📚 Documentation

Model Information

Evaluation

Evaluation Datasets Sources

• Aihub/FLoRes: traintogpb/aihub-flores-koen-integrated-sparta-30k | (test set 1k)
• iwslt-2023: shreevigneshs/iwslt-2023-en-ko-train-val-split-0.1 | (f_test 597, if_test 597)
• ko_news_2024: nayohan/ko_news_eval40 | (40)

Model Evaluation Method

• Each model is inferred based on the inference code written in the README on Hugging Face. (Common: max_new_tokens = 512)
• EEVE adds the instruction ("당신은 번역기 입니다. 영어를 한국어로 번역하세요.") to the system prompt, while KULLM3 keeps the original system prompt and adds it to the front of the user's input.

Aihub English-to-Korean Translation Dataset Evaluation

• The Aihub evaluation dataset may be included in the training datasets of the models. Please refer to it only for checking the performance by category. [Category description link] | model | aihub-111 | aihub-124 | aihub-125 | aihub-126 | aihub-563 | aihub-71265 | aihub-71266 | aihub-71382 | average | |:-----------------|------------:|------------:|------------:|------------:|------------:|--------------:|--------------:|--------------:|----------:| | EEVE-10.8b-it | 6.15 | 11.81 | 5.78 | 4.99 | 6.31 | 10.99 | 9.41 | 6.44 | 7.73 | | KULLM3 | 9.00 | 13.49 | 10.43 | 5.90 | 1.92 | 16.37 | 10.02 | 8.39 | 9.44 | | Seagull-13B | 9.8 | 18.38 | 8.51 | 5.53 | 8.74 | 17.44 | 10.11 | 11.21 | 11.21 | | Synatra-7B | 6.99 | 25.14 | 7.79 | 5.31 | 9.95 | 19.27 | 13.20 | 8.93 | 12.07 | | nhndq-nllb | 24.09 | 48.71 | 22.89 | 13.98 | 18.71 | 30.18 | 32.49 | 18.62 | 26.20 | | our-tech | 20.19 | 37.48 | 18.50 | 12.45 | 16.96 | 13.92 | 43.54 | 9.62 | 21.58 | | our-general | 24.72 | 45.22 | 21.61 | 18.97 | 17.23 | 30.00 | 32.08 | 13.55 | 25.42 | | our-sharegpt | 12.42 | 19.23 | 10.91 | 9.18 | 14.30 | 26.43 | 12.62 | 15.57 | 15.08 | | our-instrucTrans | 24.89 | 47.00 | 22.78 | 21.78 | 24.27 | 27.98 | 31.31 | 15.42 |26.92 |

FLoRes English-to-Korean Translation Dataset Evaluation

FloRes is a translation benchmark dataset publicly released by Facebook, which is constructed in parallel for English and 200 languages with few resources. The evaluation is conducted using traintogpb/aihub-flores-koen-integrated-sparta-30k. (Single-sentence composition)

iwslt-2023

The iwslt-2023 dataset consists of evaluation datasets where the same English sentences are translated into Korean in both informal and formal styles. It can relatively check the tendency of the model towards formal/informal language. (Single-sentence composition)