Radbert

🚀 RadBERT

RadBERT is a model continuously pre - trained on radiology reports, initialized from BioBERT, which can be used for tasks like fill - mask in the radiology and biomedical fields.

🚀 Quick Start

This model is suitable for fill - mask tasks, and it can infer text in scenarios such as radiology reports. For example, it can predict the masked content in text like "low lung volumes, [MASK] pulmonary vascularity."

✨ Features

• Model Type: BERT - based model, including BioBERT, RadBERT.
• Supported Tasks: fill - mask
• Libraries: pytorch, transformers
• Case Sensitivity: uncased

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

The model can be used for fill - mask tasks. For example, in a text with masked words, it can predict the appropriate content.

# There is no specific code example in the original document, here is a simple pseudo - code for reference
from transformers import AutoModelForMaskedLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('your - radbert - model')
model = AutoModelForMaskedLM.from_pretrained('your - radbert - model')

input_text = "low lung volumes, [MASK] pulmonary vascularity."
inputs = tokenizer(input_text, return_tensors='pt')
outputs = model(**inputs)

📚 Documentation

Training Data

• wikipedia
• bookscorpus
• pubmed
• radreports

Model Details

RadBERT was continuously pre - trained on radiology reports from a BioBERT initialization.

📄 License

This project is licensed under the MIT license.

📚 Citation

@article{chambon_cook_langlotz_2022, 
  title={Improved fine - tuning of in - domain transformer model for inferring COVID - 19 presence in multi - institutional radiology reports}, 
  DOI={10.1007/s10278 - 022 - 00714 - 8}, journal={Journal of Digital Imaging}, 
  author={Chambon, Pierre and Cook, Tessa S. and Langlotz, Curtis P.}, 
  year={2022}
}