Radbert RoBERTa 4m

🚀 RadBERT-RoBERTa-4m

RadBERT-RoBERTa-4m is a variant of the RadBERT models. It was trained with 4 million de-identified medical reports from US VA hospitals. This model outperforms previous medical domain models like BioBERT, Clinical-BERT, BLUE-BERT, and BioMed-RoBERTa in medical language understanding.

🚀 Quick Start

RadBERT-RoBERTa-4m is a powerful model trained on a large dataset of medical reports. It offers enhanced performance in medical language understanding compared to previous models.

✨ Features

• Stronger Medical Language Understanding: Achieves better performance than previous medical domain models.
• Evaluated on Multiple Tasks: Performance is evaluated on abnormal sentence classification, report coding, and report summarization.

Evaluation Tasks

The model's performances are evaluated on the following three tasks:

• Abnormal Sentence Classification: Classify sentences in radiology reports as reporting abnormal or normal findings.
• Report Coding: Assign a diagnostic code to a given radiology report for five different coding systems.
• Report Summarization: Given the findings section of a radiology report, extractively select key sentences that summarize the findings.

For detailed information, refer to the paper: RadBERT: Adapting transformer-based language models to radiology

The code for the paper is available at this GitHub repo.

💻 Usage Examples

Basic Usage

Here is an example of how to use this model to extract the features of a given text in PyTorch:

from transformers import AutoConfig, AutoTokenizer, AutoModel
config = AutoConfig.from_pretrained('zzxslp/RadBERT-RoBERTa-4m')
tokenizer = AutoTokenizer.from_pretrained('zzxslp/RadBERT-RoBERTa-4m')
model = AutoModel.from_pretrained('zzxslp/RadBERT-RoBERTa-4m', config=config)
text = "Replace me by any medical text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)

📄 License

BibTeX entry and citation info

If you use the model, please cite our paper:

@article{yan2022radbert,
  title={RadBERT: Adapting transformer-based language models to radiology},
  author={Yan, An and McAuley, Julian and Lu, Xing and Du, Jiang and Chang, Eric Y and Gentili, Amilcare and Hsu, Chun-Nan},
  journal={Radiology: Artificial Intelligence},
  volume={4},
  number={4},
  pages={e210258},
  year={2022},
  publisher={Radiological Society of North America}
}