Koelectra Small V3 Modu Ner

🚀 KoELECTRA-small-v3-modu-ner

This model is a fine - tuned version of monologg/koelectra-small-v3-discriminator on an unknown dataset. It can be used for token - classification tasks, achieving high performance in named entity recognition.

✨ Features

• Tagging System: It uses the BIO tagging system, which is effective in identifying named entities.
• Diverse Tag Sets: Follows the Korea Information and Communications Technology Association (TTA) classification criteria, with 15 tag sets for comprehensive entity recognition.

📦 Installation

The installation mainly involves setting up the necessary Python libraries. You can install the required libraries using pip:

pip install transformers datasets torch tokenizers

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("Leo97/KoELECTRA-small-v3-modu-ner")
model = AutoModelForTokenClassification.from_pretrained("Leo97/KoELECTRA-small-v3-modu-ner")
ner = pipeline("ner", model=model, tokenizer=tokenizer)

example = "서울역으로 안내해줘."
ner_results = ner(example)
print(ner_results)

📚 Documentation

Model description

Tagging System: BIO System

• B - (begin): Indicates the start of a named entity.
• I - (inside): Indicates that the token is in the middle of a named entity.
• O - (outside): Indicates that the token is not part of a named entity.

It follows 15 tag sets based on the classification criteria of the Korea Information and Communications Technology Association (TTA).

Intended uses & limitations

This model is mainly intended for named entity recognition tasks in Korean text. However, its performance may be affected by the domain of the text and the quality of the training data.

Training and evaluation data

The model is trained on the named entity recognition (NER) dataset from the following source:

• Ministry of Culture, Sports and Tourism > National Institute of the Korean Language > Everyone's Corpus > Named Entity Analysis Corpus 2021
• https://corpus.korean.go.kr/request/reausetMain.do

🔧 Technical Details

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 15151
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.27.4
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3