Kullm-polyglot-12.8b-v2 Open-source Model - Free-to-use Multilingual Large Language Model Supporting Korean

Home

Kullm Polyglot 12.8b V2

Developed by nlpai-lab

KULLM v2 fine-tuned version based on EleutherAI/polyglot-ko-12.8b, supporting Korean multilingual large language model

Large Language Model

Transformers

KoreanOpen Source License:Apache-2.0 #Korean optimization #Multi-GPU training #Large language model

Downloads 1,901

Release Time : 5/31/2023

Model Overview

This is a 12.8B parameter-scale multilingual large language model, specifically optimized and fine-tuned for Korean, suitable for various natural language processing tasks.

Model Features

Multilingual support

Multilingual large language model specifically optimized for Korean

Large-scale parameters

With 12.8B parameter scale, it possesses powerful language understanding and generation capabilities

Fine-tuning optimization

Specially fine-tuned and optimized based on polyglot-ko-12.8b

Model Capabilities

Text generation

Language understanding

Korean processing

Use Cases

Natural language processing

Korean text generation

Generate high-quality Korean text content

Language understanding tasks

Handle various Korean natural language understanding tasks

Property	Details
learning_rate	3e - 4
train_batch_size	6
seed	42
distributed_type	multi - GPU (A100 80G)
num_devices	4
gradient_accumulation_steps	21.3
optimizer	Adam with betas=(0.9,0.999) and epsilon=1e - 08
lr_scheduler_type	linear
num_epochs	8.0

Property	Details
Transformers	4.28.1
Pytorch	2.0.0+cu117
Datasets	2.11.0
Tokenizers	0.13.3

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご

Kullm Polyglot 12.8b V2

Model Overview

Model Features

Model Capabilities

Use Cases

🚀 KULLM-Polyglot-12.8B-v2

🚀 Quick Start

📚 Documentation

Training procedure

Training hyperparameters

Framework versions

📄 License