Llama3 Instructrans Enko 8b GGUF

🚀 llama3-instrucTrans-enko-8b-GGUF

Quantized GGUF model files for seamless English-Korean translation

This repository provides quantized GGUF model files for llama3-instrucTrans-enko-8b created by nayohan. These files are optimized for efficient inference, enabling high-quality English to Korean translation.

✨ Features

• Multilingual Support: Supports both English (en) and Korean (ko), facilitating seamless cross - language communication.
• Translation Capability: Specifically designed for English to Korean translation tasks, with high - quality generation results.
• Quantized for Efficiency: The GGUF format ensures efficient inference, making it suitable for various applications.

📦 Installation

The model can be loaded using the transformers library. Ensure you have the necessary dependencies installed:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "nayohan/llama3-instrucTrans-enko-8b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
  model_name,
  device_map="auto",
  torch_dtype=torch.bfloat16
)

💻 Usage Examples

Basic Usage

To translate text from English to Korean, use the following Python code:

system_prompt="당신은 번역기 입니다. 영어를 한국어로 번역하세요."
sentence = "The aerospace industry is a flower in the field of technology and science."
conversation = [{'role': 'system', 'content': system_prompt},
                {'role': 'user', 'content': sentence}]

inputs = tokenizer.apply_chat_template(
  conversation,
  tokenize=True,
  add_generation_prompt=True,
  return_tensors='pt'
).to("cuda")

outputs = model.generate(inputs, max_new_tokens=4096) # Finetuned with length 4096
print(tokenizer.decode(outputs[0][len(inputs[0]):]))

Example Results

# Result
INPUT: <|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n당신은 번역기 입니다. 영어를 한국어로 번역하세요.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nThe aerospace industry is a flower in the field of technology and science.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
OUTPUT: 항공우주 산업은 기술과 과학 분야의 꽃입니다.<|eot_id|>

INPUT: <|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n당신은 번역기 입니다. 영어를 한국어로 번역하세요.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n
Technical and basic sciences are very important in terms of research. It has a significant impact on the industrial development of a country. Government policies control the research budget.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
OUTPUT: 기술 및 기초 과학은 연구 측면에서 매우 중요합니다. 이는 한 국가의 산업 발전에 큰 영향을 미칩니다. 정부 정책은 연구 예산을 통제합니다.<|eot_id|>

📚 Documentation

Model Details

Evaluation

The model's performance was evaluated on multiple datasets:

• Aihub/FLoRes: traintogpb/aihub-flores-koen-integrated-sparta-30k (test set 1k)
• iwslt - 2023: shreevigneshs/iwslt-2023-en-ko-train-val-split-0.1 (f_test 597, if_test 597)
• ko_news_2024: nayohan/ko_news_eval40 (40)

Aihub English - Korean Translation Dataset Evaluation

FLoRes English - Korean Translation Dataset Evaluation

iwslt - 2023 Evaluation

📄 License

This model is released under the llama3 license. Please review the license terms before use.