K2S3 SOLAR 11b V2.0

🚀 K2S3-SOLAR-11b-v1.0

This is a fine - tuned model based on the upstage/SOLAR - 10.7B - v1.0, aiming to better adapt to Korean - related tasks.

🚀 Quick Start

This section provides an overview of the K2S3 - SOLAR - 11b - v1.0 model, including its development information, base model, training data, training method, and hardware used.

✨ Features

• Based on a strong foundation: Built upon the upstage/SOLAR - 10.7B - v1.0 base model.
• Rich training data: Trained with a variety of Korean - related data sources.
• Full - parameter fine - tuning: Fine - tuned using the SFT method with full - parameter adjustment.

📚 Documentation

Developed by

• K2S3

Model Number

• K2S3 - SOLAR - 11b - v1.0

Base Model

• [upstage/SOLAR - 10.7B - v1.0](https://huggingface.co/upstage/SOLAR - 10.7B - v1.0)

Training Data

The training data for this model includes the Standard Korean Dictionary, training data from KULLM at Korea University, abstracts of master's and doctoral theses, and Korean language samples from AI Hub.

Training Method

This model was fine - tuned on the "upstage/SOLAR - 10.7B - v1.0" base model using a full parameter tuning method with SFT (Supervised Fine - Tuning).

Hardware

• Hardware: Utilized two A100 (80G*2EA) GPUs for training.
• Training Factors: This model was fine - tuned with SFT, using the HuggingFace SFTtrainer and applied fsdp. Key training adjustments include the addition of new Korean tokens trained with the SentencePieceBPETokenizer, trained for 2 epochs, batch size of 1, and gradient accumulation of 32.

📄 License

This model is licensed under the CC - BY - NC - 4.0 license.