Surgicberta

🚀 SurgicBERTa

SurgicBERTa is a language model adapted from RoBERTa-base for surgical language understanding, trained on a large corpus of surgical textbooks and academic papers.

🚀 Quick Start

SurgicBERTa is a language model based on RoBERTa-base (Liu et al., 2019) architecture. We adapted RoBERTa-base to different surgical textbooks and academic papers via continued pretraining. This amounts to about 7 million words and 300k surgical sentences. We used the full text of the books and papers in training, not just abstracts. Specific details of the adaptive pretraining procedure and evaluation tasks can be found in the paper below cited.

📦 Widget Examples

Here are some examples to show the application scenarios of SurgicBERTa:

• Example 1: "hemithyroidectomy is the removal of half of the gland."
• Example 2: "Li - Fraumeni is a hereditary tumor with autosomal dominant inheritance."
• Example 3: "The fascia in front of the pancreas was cut to the spleen direction to the splenic artery and vein."

📄 License

This project is licensed under the CC BY - NC - ND 4.0 license.

📚 Citation

General Use

If using this model, please cite the following paper:

@article{bombieri_et_al_SurgicBERTa_2023,
title = {Surgicberta: a pre-trained language model for procedural surgical language},
journal = {International Journal of Data Science and Analytics},
year = {2023},
doi = { https://doi.org/10.1007/s41060-023-00433-5 },
url = { https://link.springer.com/article/10.1007/s41060-023-00433-5 },
author = {Marco Bombieri and Marco Rospocher and Simone Paolo Ponzetto and Paolo Fiorini},
}

For Semantic Role Labeling

If using this model for Semantic Role Labeling, please cite also the following paper:

@article{bombieri_et_al_surgical_srl_2023,
title = {Machine understanding surgical actions from intervention procedure textbooks},
journal = {Computers in Biology and Medicine},
volume = {152},
pages = {106415},
year = {2023},
issn = {0010-4825},
doi = {https://doi.org/10.1016/j.compbiomed.2022.106415},
url = {https://www.sciencedirect.com/science/article/pii/S0010482522011234},
author = {Marco Bombieri and Marco Rospocher and Simone Paolo Ponzetto and Paolo Fiorini},
keywords = {Semantic role labeling, Surgical data science, Procedural knowledge, Information extraction, Natural language processing}
}