Biomednlp BiomedBERT Base Uncased Abstract

🚀 MSR BiomedBERT (abstracts only)

このモデルは、大規模なニューラル言語モデルで、PubMedの論文の要約を使って学習されています。生物医学分野の自然言語処理タスクで高い性能を発揮します。

🚀 クイックスタート

Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. Recent work shows that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.

This BiomedBERT is pretrained from scratch using abstracts from PubMed. This model achieves state-of-the-art performance on several biomedical NLP tasks, as shown on the Biomedical Language Understanding and Reasoning Benchmark.

📄 ライセンス

このモデルはMITライセンスの下で提供されています。

📚 ドキュメント

引用

もしあなたの研究でBiomedBERTが役立った場合は、以下の論文を引用してください。

@misc{pubmedbert,
  author = {Yu Gu and Robert Tinn and Hao Cheng and Michael Lucas and Naoto Usuyama and Xiaodong Liu and Tristan Naumann and Jianfeng Gao and Hoifung Poon},
  title = {Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing},
  year = {2020},
  eprint = {arXiv:2007.15779},
}