Gliner Ko

🚀 GLiNER-ko Model Card

GLiNER is a Named Entity Recognition (NER) model. It uses a bidirectional transformer encoder (similar to BERT) to identify any entity type. This offers a practical alternative to traditional NER models, which are restricted to predefined entities, and Large Language Models (LLMs). LLMs are flexible but costly and large, making them unsuitable for resource - constrained scenarios.

This version is trained on the various Korean NER dataset (for research purposes). Commercially - permitted versions are available, including urchade/gliner_smallv2, urchade/gliner_mediumv2, and urchade/gliner_largev2.

🚀 Quick Start

GLiNER is a powerful tool for Named Entity Recognition. It can identify various entity types, providing a great solution for NER tasks.

✨ Features

• Capable of identifying any entity type using a bidirectional transformer encoder.
• A practical alternative to traditional NER models and LLMs in resource - constrained scenarios.
• Different versions are available for both research and commercial use.

📦 Installation

To use this model, you must install the Korean fork of GLiNER Python library and mecab - ko:

!pip install gliner
!pip install python-mecab-ko

💻 Usage Examples

Basic Usage

Once you've downloaded the GLiNER library, you can import the GLiNER class. You can then load this model using GLiNER.from_pretrained and predict entities with predict_entities.

from gliner import GLiNER

model = GLiNER.from_pretrained("taeminlee/gliner_ko")

text = """
피터 잭슨 경(, 1961년 10월 31일 ~ )은 뉴질랜드의 영화 감독, 각본가, 영화 프로듀서이다. J. R. R. 톨킨의 소설을 원작으로 한 《반지의 제왕 영화 3부작》(2001년~2003년)의 감독으로 가장 유명하다. 2005년에는 1933년작 킹콩의 리메이크작 《킹콩(2005)》의 감독을 맡았다.
"""

tta_labels = ["ARTIFACTS", "ANIMAL", "CIVILIZATION", "DATE", "EVENT", "STUDY_FIELD", "LOCATION", "MATERIAL", "ORGANIZATION", "PERSON", "PLANT", "QUANTITY", "TIME", "TERM", "THEORY"]

entities = model.predict_entities(text, labels)

for entity in entities:
    print(entity["text"], "=>", entity["label"])

피터 잭슨 경 => PERSON
1961년 10월 31일 ~ => DATE
뉴질랜드 => LOCATION
영화 감독 => CIVILIZATION
각본가 => CIVILIZATION
영화 => CIVILIZATION
프로듀서 => CIVILIZATION
J. R. R. 톨킨 => PERSON
3부작 => QUANTITY
2001년~2003년 => DATE
감독 => CIVILIZATION
2005년 => DATE
1933년작 => DATE
킹콩 => ARTIFACTS
킹콩 => ARTIFACTS
2005 => DATE
감독 => CIVILIZATION

📚 Documentation

Links

• Paper: https://arxiv.org/abs/2311.08526
• Repository: https://github.com/urchade/GLiNER

Named Entity Recognition benchmark result

Evaluate with the konne dev set

🔧 Technical Details

GLiNER uses a bidirectional transformer encoder (BERT - like) to identify any entity type. It provides a practical alternative to traditional NER models and LLMs, especially in resource - constrained scenarios.

📄 License

The model is licensed under cc - by - nc - 4.0.

Model Authors

The model authors are:

• Taemin Lee
• Urchade Zaratiana
• Nadi Tomeh
• Pierre Holat
• Thierry Charnois

Citation

@misc{zaratiana2023gliner,
      title={GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer}, 
      author={Urchade Zaratiana and Nadi Tomeh and Pierre Holat and Thierry Charnois},
      year={2023},
      eprint={2311.08526},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}