🚀 BioELECTRA - PICO
BioELECTRA - PICO 是一个针对生物医学领域的预训练文本编码器模型,它采用了 ELECTRA 的 “替换词检测” 预训练技术,在多个生物医学 NLP 基准测试中表现出色,为生物医学文本挖掘任务提供了强大的支持。
🚀 快速开始
引用说明
如果您使用了我们的研究成果,请使用以下 BibTeX 格式引用我们的论文:
@inproceedings{kanakarajan-etal-2021-bioelectra,
title = "{B}io{ELECTRA}:Pretrained Biomedical text Encoder using Discriminators",
author = "Kanakarajan, Kamal raj and
Kundumani, Bhuvana and
Sankarasubbu, Malaikannan",
booktitle = "Proceedings of the 20th Workshop on Biomedical Language Processing",
month = jun,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.bionlp-1.16",
doi = "10.18653/v1/2021.bionlp-1.16",
pages = "143--154",
abstract = "Recent advancements in pretraining strategies in NLP have shown a significant improvement in the performance of models on various text mining tasks. We apply {`}replaced token detection{'} pretraining technique proposed by ELECTRA and pretrain a biomedical language model from scratch using biomedical text and vocabulary. We introduce BioELECTRA, a biomedical domain-specific language encoder model that adapts ELECTRA for the Biomedical domain. WE evaluate our model on the BLURB and BLUE biomedical NLP benchmarks. BioELECTRA outperforms the previous models and achieves state of the art (SOTA) on all the 13 datasets in BLURB benchmark and on all the 4 Clinical datasets from BLUE Benchmark across 7 different NLP tasks. BioELECTRA pretrained on PubMed and PMC full text articles performs very well on Clinical datasets as well. BioELECTRA achieves new SOTA 86.34{\%}(1.39{\%} accuracy improvement) on MedNLI and 64{\%} (2.98{\%} accuracy improvement) on PubMedQA dataset.",
}
示例信息
在相关研究中发现:与安慰剂组相比,阿司匹林组的头痛持续时间有所缩短(P<0.05)。
widget:
- text: "Those in the aspirin group experienced reduced duration of headache compared to those in the placebo arm (P<0.05)"