Clinical Pubmed Bert Base 512

🚀 ClinicalPubMedBERT

A pre - trained model for clinical decision support, aiming to achieve better results on clinical - related downstream tasks by combining EHRs and biomedical papers.

🚀 Quick Start

For more details about this model, please visit https://github.com/NtaylorOX/Public_Prompt_Mimic_III.

✨ Features

• It is a BERT model pre - trained on PubMed abstracts and continually pre - trained on clinical notes (MIMIC - III). By combining two domains (EHRs and biomedical papers) with fewer overlaps with general knowledge text corpora, it hopes to serve better results on clinical - related downstream tasks such as readmissions.
• The model is trained on 500000 clinical notes randomly sampled from MIMIC datasets, with 100k steps of training.
• Whole word masking is used to enhance the coherence of the language model.
• All notes are chunked into a length of 512 tokens.

💻 Usage Examples

Basic Usage

Here are some examples of using the model:

Example 1

Due to shortness of breath, the patient is diagnosed with [MASK], and other respiratory problems.

Example 2

Due to high blood sugar, and very low blood pressure, the patient is diagnosed with [MASK].

📚 Documentation

The pre - trained model can be accessed at https://huggingface.co/microsoft/BiomedNLP - PubMedBERT - base - uncased - abstract.

📄 License

This project is licensed under the MIT license.

📦 Installation

No installation steps were provided in the original document, so this section is skipped.

🔧 Technical Details

No specific technical details (more than 50 words) were provided in the original document, so this section is skipped.