Llama3.1 Korean V1.1 Sft By Aidx

🚀 ktdsbaseLM-v0.14-onbased-llama3.1

This model is based on LlaMA3.1 and is developed to be applicable to Korean and various Korean cultures. It uses self - produced Korean data from 53 domains to understand Korean social values and cultures.

✨ Features

• Multifunctional: Supports various tasks such as text generation, dialogue inference, document summarization, question - answering, sentiment analysis, and other natural language processing - related tasks. It can be applied in diverse fields like law, finance, science, education, business, and cultural research.
• Korean - specific: Designed to understand the Korean language and various Korean cultural contexts. It reflects the values and cultures of Korean society by leveraging self - produced Korean data from 53 domains.
• Efficient Architecture: Based on the LlaMA3.1 8B model with 8 billion parameters. Its lightweight architecture ensures fast inference speed and memory efficiency, and is optimized for various natural language processing tasks.

📦 Installation

No specific installation steps are provided in the original document.

📚 Documentation

Model Information

Model Description

This model is fine - tuned using the SFT method based on the LlaMA3.1 model. It is designed to understand the Korean language and various Korean cultural contexts, and reflects the values and cultures of Korean society by using self - produced Korean data from 53 domains.

Training Data

• The model is trained on a total of 3.6GB of self - developed data, including 2.33 million pieces of data such as Q&A, summarization, and classification. Among them, 1.33 million are multiple - choice questions from 53 domains (including Korean history, society, finance, law, taxation, mathematics, biology, physics, chemistry, etc.) and are trained using the Chain of Thought method. 1.3 million subjective questions are trained across 38 domains such as Korean history, finance, law, taxation, and mathematics. The model has learned data that can understand Korean social values and human emotions and output according to the given instructions.
• Training Instruction Datasets Format:

  {"prompt": "prompt text", "completion": "ideal generated text"}

💻 Usage Examples

Basic Usage

The model can be used in various application scenarios:

• Education: Generate question - answers and explanations for various learning materials in history, mathematics, science, etc.
• Business: Provide answers to legal, financial, and tax - related questions and summarize documents.
• Research and Culture: Perform natural language processing tasks, sentiment analysis, document generation, and translation in line with Korean society and culture.
• Customer Service: Generate conversations with users and provide customized responses.

Code Example

from transformers import AutoModel, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("AIDX-ktds/ktdsbaseLM-v0.14-onbased-llama3.1")
model = AutoModel.from_pretrained("AIDX-ktds/ktdsbaseLM-v0.14-onbased-llama3.1")

input_text =  """ 「국민건강보험법」제44조, 「국민건강보험법 시행령」제19조,「약관의 규제에 관한 법률」제5조, 「상법」제54조 참조 판단 해줘""" + " 답변:"
inputs = tokenizer(input_text, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(**inputs, max_length=1024,  temperature=0.5, do_sample=True, repetition_penalty=1.15)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

🔧 Technical Details

The model is based on the LlaMA3.1 8B model, with 8 billion parameters. It uses the SFT method for fine - tuning, which is optimized for Korean language and culture - specific tasks. The lightweight architecture of LlaMA3.1 8B ensures fast inference speed and memory efficiency, making it suitable for a variety of natural language processing tasks.

⚠️ Limitations

• Since the model is specialized in the Korean language and Korean culture, the accuracy of responses in other languages or cultures may be affected due to the lack of data in specific areas (e.g., the latest international materials, professional fields).
• It may show limited inference ability for problems that require complex logical thinking, and there is a possibility of generating biased responses if the training data contains biases.

📄 License

The model is released under the apache - 2.0 license.