KR FinBert SC

🚀 KR-FinBert & KR-FinBert-SC

In the field of Natural Language Processing (NLP), significant progress has been made. Many studies have demonstrated that domain adaptation using a small - scale corpus and fine - tuning with labeled data can effectively improve overall performance. We proposed KR - FinBert for the financial domain by further pre - training it on a financial corpus and fine - tuning it for sentiment analysis. As shown in many studies, the performance improvement through adaptation and downstream tasks was also evident in this experiment.

🚀 Quick Start

In the NLP (Natural Language Processing) field, substantial progress has been achieved. A multitude of studies indicate that domain adaptation with a small - scale corpus and fine - tuning using labeled data are effective for enhancing overall performance. We introduced KR - FinBert for the financial domain. This was accomplished by further pre - training it on a financial corpus and fine - tuning it for sentiment analysis. Similar to many other studies, the performance improvement through adaptation and downstream tasks was also clear in this experiment.

✨ Features

• Domain - Specific Adaptation: KR - FinBert is pre - trained on a financial corpus, which is beneficial for financial - related NLP tasks.
• Sentiment Analysis Fine - Tuning: Fine - tuned for sentiment analysis, it can accurately classify financial texts' sentiment.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

📚 Documentation

Data

The training data for this model is an expansion of that of [KR - BERT - MEDIUM](https://huggingface.co/snunlp/KR - Medium), including texts from Korean Wikipedia, general news articles, legal texts crawled from the National Law Information Center, and the [Korean Comments dataset](https://www.kaggle.com/junbumlee/kcbert - pretraining - corpus - korean - news - comments). For transfer learning, corporate - related economic news articles from 72 media sources such as the Financial Times, The Korean Economy Daily, etc., and analyst reports from 16 securities companies such as Kiwoom Securities, Samsung Securities, etc. are added. The dataset contains 440,067 news titles with their content and 11,237 analyst reports. The total data size is approximately 13.22GB. For mlm training, we split the data line by line, and the total number of lines is 6,379,315. KR - FinBert is trained for 5.5M steps with a maxlen of 512, a training batch size of 32, and a learning rate of 5e - 5. It takes 67.48 hours to train the model using an NVIDIA TITAN XP.

Downstream tasks

Sentimental Classification model

The downstream task performances are evaluated with 50,000 labeled data.

Inference sample

Citation

@misc{kr-FinBert-SC,
  author = {Kim, Eunhee and Hyopil Shin},
  title = {KR-FinBert: Fine-tuning KR-FinBert for Sentiment Analysis},
  year = {2022},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://huggingface.co/snunlp/KR-FinBert-SC}}
}