Mbg Clinicalbert

🚀 MBG-ClinicalBERT

MBG-ClinicalBERT是一个基于ClinicalBERT的模型，并在保加利亚医学和临床文本上进行了额外的预训练。它能够更好地处理保加利亚语的医学和临床文本，为相关领域的自然语言处理任务提供支持。

📚 详细文档

模型详情

引用方式

@inproceedings{velichkov-etal-2021-comparative,
    title = "Comparative Analysis of Fine-tuned Deep Learning Language Models for {ICD}-10 Classification Task for {B}ulgarian Language",
    author = "Velichkov, Boris  and
      Vassileva, Sylvia  and
      Gerginov, Simeon  and
      Kraychev, Boris  and
      Ivanov, Ivaylo  and
      Ivanov, Philip  and
      Koychev, Ivan  and
      Boytcheva, Svetla",
    booktitle = "Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)",
    month = sep,
    year = "2021",
    address = "Held Online",
    publisher = "INCOMA Ltd.",
    url = "https://aclanthology.org/2021.ranlp-1.162",
    pages = "1448--1454",
    abstract = "The task of automatic diagnosis encoding into standard medical classifications and ontologies, is of great importance in medicine - both to support the daily tasks of physicians in the preparation and reporting of clinical documentation, and for automatic processing of clinical reports. In this paper we investigate the application and performance of different deep learning transformers for automatic encoding in ICD-10 of clinical texts in Bulgarian. The comparative analysis attempts to find which approach is more efficient to be used for fine-tuning of pretrained BERT family transformer to deal with a specific domain terminology on a rare language as Bulgarian. On the one side are used SlavicBERT and MultiligualBERT, that are pretrained for common vocabulary in Bulgarian, but lack medical terminology. On the other hand in the analysis are used BioBERT, ClinicalBERT, SapBERT, BlueBERT, that are pretrained for medical terminology in English, but lack training for language models in Bulgarian, and more over for vocabulary in Cyrillic. In our research study all BERT models are fine-tuned with additional medical texts in Bulgarian and then applied to the classification task for encoding medical diagnoses in Bulgarian into ICD-10 codes. Big corpora of diagnosis in Bulgarian annotated with ICD-10 codes is used for the classification task. Such an analysis gives a good idea of which of the models would be suitable for tasks of a similar type and domain. The experiments and evaluation results show that both approaches have comparable accuracy.",
}

📄 许可证

本项目采用AFL-3.0许可证。