Llama3.1 Korean V1.6 Sft By Aidx

🚀 Korean Language Model Based on Llama 3.1

This model is developed based on Llama 3.1, fine - tuned with Korean data to understand Korean language and culture, and can be applied to various natural language processing tasks.

🚀 Quick Start

Basic Usage

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("SEOKDONG/llama3.0_korean_v1.0_sft")
model = AutoModel.from_pretrained("SEOKDONG/llama3.0_korean_v1.0_sft")

input_text =  """ 「국민건강보험법」제44조, 「국민건강보험법 시행령」제19조,「약관의 규제에 관한 법률」제5조, 「상법」제54조 참조 판단 해줘""" + " 답변:"
inputs = tokenizer(input_text, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(**inputs, max_length=1024,  temperature=0.5, do_sample=True, repetition_penalty=1.15)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

✨ Features

This model is developed based on Llama 3.1 and fine - tuned using the SFT method. It is designed to understand the Korean language and various cultural contexts in Korea. By leveraging self - produced Korean data from 53 domains, it reflects the values and culture of Korean society. Its main functions include text generation, dialogue reasoning, document summarization, question - answering, sentiment analysis, and various other natural - language - processing - related tasks. It can be applied in various fields such as law, finance, science, education, business, and cultural research.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

The model can be used in a wide range of application fields, for example:

• Education: Generate question - answering and explanations for various learning materials in history, mathematics, science, etc.
• Business: Provide answers to legal, financial, and tax - related questions and summarize documents.
• Research and Culture: Perform natural - language - processing tasks tailored to Korean society and culture, sentiment analysis, document generation, and translation.
• Customer Service: Generate conversations with users and provide customized responses.

📚 Documentation

Model Description

• Model Name and Main Functions: This model is fine - tuned using the SFT method based on the Llama 3.1 model. It is designed to understand the Korean language and various cultural contexts in Korea. By using self - produced Korean data from 53 domains, it reflects the values and culture of Korean society. Its main functions support various tasks related to natural language processing, and it can be applied in multiple fields.
• Model Architecture: Based on the Llama 3.1 8B model, this high - performance language model consists of 8 billion parameters (8B). Using Llama 3.1 8B as the foundation model, it is trained through the SFT (Supervised Fine - Tuning) method to perform specifically for the Korean language and culture. The lightweight structure of Llama 3.1 8B ensures fast inference speed and memory efficiency and is optimized for various natural - language - processing tasks.

Training Data

• This model is trained on a self - developed dataset with a total size of 3.6GB, including a total of 2.33 million pieces of data such as Q&A, summarization, and classification. Among them, 1.33 million are multiple - choice questions from 53 domains, including Korean history, society, finance, law, taxation, mathematics, biology, physics, chemistry, etc., and are trained using the Chain of Thought method. Also, 1.3 million subjective questions are trained across 38 domains such as Korean history, finance, law, and taxation. The training data includes data that can understand Korean social values and human emotions and output according to instructions.
• Training Instruction Datasets Format:

{"prompt": "prompt text", "completion": "ideal generated text"}

Limitations

Although this model is specialized for the Korean language and culture, due to the lack of data in specific domains (e.g., the latest international materials, professional fields), the accuracy of responses in other languages or cultures may be low. Also, it may show limited reasoning ability for problems that require complex logical thinking, and there is a possibility of generating biased responses if the data contains biases.

🔧 Technical Details

The model is based on the Llama 3.1 8B model, with 8 billion parameters. It is fine - tuned using the SFT method with self - produced Korean data from 53 domains to adapt to the Korean language and culture. The lightweight architecture of Llama 3.1 8B ensures fast inference speed and memory efficiency, making it suitable for various natural - language - processing tasks.

📄 License

This model is licensed under the Apache 2.0 license.