Km Bert

🚀 Model Card for Model ID

This model is a conversion of the KM - BERT model, making it more convenient to use in the form of a Hugging Face model.

This model keeps the original architecture, and for the tokenizer, we've adopted the snunlp/KR - BERT - char16424 as described in the paper.

We're grateful to the authors who shared the KM - BERT model, enabling us to try it out.

If any issues arise, we may make modifications. Also, upon the authors' request, we may remove it from the hub.

📚 Documentation

Summary using Bing:

This document covers the following aspects:

A pre - trained BERT model for Korean medical natural language processing. This model, called KM - BERT, is pre - trained on a Korean medical corpus based on the KR - BERT model to analyze Korean texts used in the medical field. It uses a Korean medical corpus of about 111.6 million words, which consists of three types of documents: medical textbooks, health information news, and medical research papers.

Internal and external evaluations: This model performs two unsupervised pre - training tasks, MLM and NSP, and internally evaluates its language understanding ability using a new Korean medical corpus and the MedSTS dataset. Additionally, it externally evaluates its performance on downstream tasks using the Korean medical semantic text similarity (MedSTS) dataset and the Korean medical named - entity recognition (NER) dataset. As a result, KM - BERT shows better performance than existing language models.

Perspectives by language and field: Although M - BERT shows high transferability for NLP tasks across various languages, it has been proven that language - specific BERT models perform better than M - BERT.

1. Since Korean has a different word order from Indo - European languages like English, a Korean - specific BERT model is necessary.
2. It has also been found that field - specific BERT embeddings, such as SciBERT and LEGAL - BERT, improve the understanding of technical terms and usages compared to BERT.
3. These research results demonstrate the necessity and effectiveness of developing a BERT model suitable for the Korean medical field.

Model Sources [optional]

• Repository: https://github.com/KU-RIAS/KM-BERT-Korean-Medical-BERT
• Paper: https://www.nature.com/articles/s41598-022-17806-8

📄 License

Citation [optional]

@article{KMBERT,
    title={KM-BERT: A Pre-trained BERT for Korean Medical Natural Language Processing},
    author={TBD},
    year={TBD},
    journal={TBD},
    volume={TBD}
  }