Llama 3.1 Korean 8B Instruct

🚀 Llama-3.1-Korean-8B-Instruct

Llama-3.1-Korean-8B-Instruct is a fine-tuned model based on Meta-Llama-3.1, designed to handle Korean language tasks effectively.

Tags

• llama
• sh2orc

Base Model

• meta-llama/Meta-Llama-3.1-8B-Instruct

Dataset

• maywell/ko_wikidata_QA
• lcw99/wikipedia-korean-20240501-1million-qna
• jojo0217/korean_rlhf_dataset
• MarkrAI/KoCommercial-Dataset

🚀 Quick Start

💻 Usage for Transformers

Starting with transformers >= 4.43.0, you can run conversational inference using the Transformers pipeline abstraction or by leveraging the Auto classes with the generate() function. Make sure to update your transformers installation via pip install --upgrade transformers.

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "sh2orc/Llama-3.1-Korean-8B-Instruct"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=2048, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

💻 Usage for VLLM

Starting with vllm, you can run conversational inference using the vLLM pipeline abstraction with the gen() function. Make sure to update your vllm installation via pip install --upgrade vllm.

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer, pipeline

BASE_MODEL = "sh2orc/Llama-3.1-Korean-8B-Instruct"

llm = LLM(model=BASE_MODEL)

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = 'right'

def gen(instruction):
    messages = [
        {
          "role": "system",
          "content": "당신은 훌륭한 AI 비서입니다. 답변 중 모르는 내용이면 모른다고 답해주세요. You are a great AI assistant. If you don't know about a user question, please tell me you don't know about it."
        },
        {
          "role": "user",
          "content": instruction
        }, 
    ]
    
    prompt_message = tokenizer.apply_chat_template(
            messages, 
            tokenize=False, 
            add_generation_prompt=True,
    )
    
    eos_token_id = [tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|eot_id|>")]
    
    outputs = llm.generate(prompt_message, SamplingParams(
        stop_token_ids=eos_token_id, 
        temperature=0.3, 
        top_p=0.9,
        frequency_penalty=1.3,
        max_tokens=4096))
    
    for output in outputs:
        propt = output.prompt
        generated_text = output.outputs[0].text
        print(generated_text)

❓ Q&A

Q. Can you tell me about Na Hye-seok?

A. Na Hye-seok (October 28, 1896 - February 1948) was a Korean female writer and social activist, known as a representative feminist writer during the Japanese colonial period. She received her education in Japan and participated in the Japanese feminist movement. In Korea, she founded the Joseon Women's Education Association and participated in the movement to establish women's schools.

Na Hye-seok was active in various genres such as poetry and novels, and her works mainly dealt with women's lives and social issues. Her representative works include 'Arirang', 'Salt', 'Pigeon', etc. Among them, 'Arirang' is her most famous work.

Na Hye-seok was also well-known as a political figure during the Japanese colonial period. She supported the independence movement and participated in the anti-Japanese movement. After the establishment of the Republic of Korea government, she continued her political activities.

Her life and achievements are evaluated as having played an important role in the history of modern Korean literature and feminism.

Q. Can you tell me about World War II?

A. World War II was a large-scale military conflict that took place from 1939 to 1945 and was the first global war after World War I. This war was initiated mainly by Germany and Italy, and the Soviet Union and the United States participated as major Allied powers. Japan, although initially neutral, formed an alliance with Germany and invaded Manchuria and China.

Germany, under the rule of Adolf Hitler of the Nazi Party, pursued an expansionist policy, annexed Austria, and partitioned Czechoslovakia. France invaded Poland but was defeated, and Britain also surrendered to Germany. However, in Britain, the resistance movement became active after Winston Churchill came to power, and in the United States, President Franklin D. Roosevelt's re - election led to its participation in the war.

The Soviet Union, under Stalin's leadership, pursued an independent foreign policy and formed an alliance with Japan. After winning the war against Nazi Germany, it exerted influence across Europe. The United States and Britain won the Battle of the Atlantic and advanced into the European mainland, and the Soviet Union also advanced to the Vistula - Oder line.

Japan aimed to expand its influence in the Pacific region and exerted influence on the Chinese mainland and the Korean Peninsula. However, it was defeated after the failure of its landing operations due to the US military attacks.

Many countries participated in World War II, and the number of casualties was approximately 50 - 80 million. As a result of this war, the United Nations organization was established, which is known as the United Nations.

Of course, this may not be all there is to know. If you want more detailed information, please feel free to ask additional questions or request more materials!

Q. What happens if I borrow money and don't pay it back?

A. If you borrow money and don't pay it back, serious problems may occur. First, a contract must be made between the borrower and the lender. This contract is legally recognized, and it specifies the loan amount, interest rate, repayment period, etc.

If you default on the loan, the following consequences may occur:

1. Interest Charges: Interest will be charged on the overdue amount. The interest is calculated as a certain percentage of the principal.
2. Default on Debt: The borrower will be in violation of the debt - repayment obligation.
3. Legal Action: The creditor can file a lawsuit in court to demand repayment or claim compensation.
4. Credit Score Drop: If you have a credit evaluation, your credit score will drop, and you may face difficulties in applying for loans or getting credit cards from financial institutions.

Therefore, you must repay the borrowed money. If you are unable to pay, you should immediately notify the lender and negotiate a solution.

However, since every situation is different, it may be necessary to seek professional or legal advice.

Do you have any other questions?

Q. How to translate the following sentence into English: "To live a healthy life, it is important to maintain a balanced diet, do regular exercise, get enough sleep, manage stress, and have regular health check - ups to detect and manage diseases early."

A. Here are some English translations:

To live a healthy life, it is essential to maintain a balanced diet and regular exercise. Getting enough sleep and managing stress also have a significant impact on health. Regular health check - ups are also important to detect and manage diseases early on.

Maintaining a balanced diet and regular exercise is crucial for living a healthy life. Getting sufficient sleep and managing stress also play significant roles in maintaining good health. Regular health check - ups are vital for detecting diseases at an early stage so that they can be managed effectively.

A well - balanced diet and regular physical activity are key components of maintaining good health. Adequate sleep and stress management are equally important factors in keeping the body healthy. Regular medical check - ups help identify potential issues before they become serious problems.

There are also many other ways to translate it into English.

Q. Can you translate the following sentence into English: "Credit card delinquency can cause significant financial disadvantages. When delinquency occurs, the credit rating drops significantly, which may result in rejection or unfavorable terms when applying for loans later. Also, there is a high possibility of receiving high - interest loan or other debt proposals from the credit card company."

A. Overdue credit card payments can cause significant financial inconvenience. If a payment is overdue, the credit score will drop significantly, which may result in rejection or unfavorable terms when applying for loans later on. Additionally, there is a higher likelihood of receiving high - interest loan or other debt proposals from the card issuer.