Llama3.1 Korean V0.1 Sft By Aidx

🚀 Llama3.1 Korean Model

This model is developed based on Llama3.1 Instruct, aiming to be applicable to Korean language and various Korean cultures. It utilizes self - produced Korean data from 53 domains to understand Korean social values and cultures. Thanks for ktds.

📚 Documentation

Model Overview

This model is built upon the Llama3.1 Instruct as its foundation model. It is developed to be applicable to the Korean language and various aspects of Korean culture. It is a model that understands Korean social values and cultures by leveraging self - produced Korean data from 53 domains.

Training Data

• This model is trained on a self - developed dataset with a total size of 3.6GB. It includes a total of 2.33 million pieces of data such as Q&A, summarization, and classification. Among them, 1.33 million pieces are composed of multiple - choice questions from 53 domains, including Korean history, society, finance, law, taxation, mathematics, biology, physics, chemistry, etc., and are trained in the Chain of Thought manner. Also, 1.3 million pieces of subjective questions are trained across 38 domains such as Korean history, finance, law, taxation, and mathematics. The model has learned data that can understand Korean social values and human emotions and generate outputs according to given instructions.
• Training Instruction Datasets Format:

{"prompt": "prompt text", "completion": "ideal generated text"}

Use Cases

This model can be used in a variety of application fields. For example:

• Education: Generating Q&A and explanations for various learning materials in history, mathematics, science, etc.
• Business: Providing answers to legal, financial, and tax - related inquiries and summarizing documents.
• Research and Culture: Natural language processing tasks tailored to Korean society and culture, sentiment analysis, document generation, and translation.
• Customer Service: Generating conversations with users and providing customized responses. This model has high applicability in various natural language processing tasks.

Limitations

Although this model is specialized in the Korean language and Korean culture, due to the lack of data in specific domains (e.g., the latest international materials, professional fields), the accuracy of responses to other languages or cultures may be low. Also, it may show limited reasoning ability for problems that require complex logical thinking, and there is a possibility of generating biased responses if the training data contains biases.

Usage Examples

Basic Usage

from transformers import AutoModel, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("SEOKDONG/llama3.1_korean_v0.1_sft_by_aidx")
model = AutoModel.from_pretrained("SEOKDONG/llama3.1_korean_v0.1_sft_by_aidx")

input_text =  """ 「국민건강보험법」제44조, 「국민건강보험법 시행령」제19조,「약관의 규제에 관한 법률」제5조, 「상법」제54조 참조 판단 해줘"""
inputs = tokenizer(input_text, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(**inputs, max_length=1024,  temperature=0.5, do_sample=True, repetition_penalty=1.15)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

📄 License

This model is under the Apache - 2.0 license.

📋 Information Table