Roberta Es Clinical Trials Temporal Ner

🚀 roberta-es-clinical-trials-temporal-ner

This named entity recognition model detects temporal expressions (TIMEX) according to the TimeML scheme (Pustejovsky et al. 2005), in addition to Age entities. It offers significant value in the field of clinical trials by accurately identifying key temporal and age - related information in Spanish texts.

✨ Features

• Entity Detection: Capable of detecting various types of entities, including Age (e.g., 18 años), Date (e.g., 2022, 26 de noviembre), Duration (e.g., 3 horas), Frequency (e.g., semanal), and Time (e.g., noche).
• High - Performance Results: Achieves excellent performance on the test set, with Precision, Recall, F1, and Accuracy all reaching around 0.9, and an extremely high accuracy of 0.996 (±0.001).

📦 Installation

No installation steps are provided in the original README, so this section is skipped.

📚 Documentation

Model description

This model adapts the pre - trained model bsc-bio-ehr-es, presented in Pio Carriño et al. (2022). It is fine - tuned to conduct temporal named entity recognition on Spanish texts about clinical trials. The model is fine - tuned on the CT - EBM - ES corpus (Campillos - Llanos et al. 2021).

If you use this model, please, cite as follows:

@article{campillosetal2024,
        title = {{Hybrid tool for semantic annotation and concept extraction of medical texts in Spanish}},
        author = {Campillos - Llanos, Leonardo and Valverde - Mateos, Ana and Capllonch - Carri{\'o}n, Adri{\'a}n},
        journal = {BMC Bioinformatics},
        year={2024},
        publisher={BioMed Central}
}

Intended uses & limitations

⚠️ Important Note

This model is under development and needs to be improved. It should not be used for medical decision making without human assistance and supervision.

This model is intended for a generalist purpose, and may have bias and/or any other undesirable distortions.

Third parties who deploy or provide systems and/or services using any of these models (or using systems based on these models) should note that it is their responsibility to mitigate the risks arising from their use. Third parties, in any event, need to comply with applicable regulations, including regulations concerning the use of artificial intelligence.

The owner or creator of the models will in no event be liable for any results arising from the use made by third parties of these models.

Training and evaluation data

The data used for fine - tuning are the Clinical Trials for Evidence - Based - Medicine in Spanish corpus. It is a collection of 1200 texts about clinical trials studies and clinical trials announcements:

• 500 abstracts from journals published under a Creative Commons license, e.g., available in PubMed or the Scientific Electronic Library Online (SciELO)
• 700 clinical trials announcements published in the European Clinical Trials Register and Repositorio Español de Estudios Clínicos

If you use the CT - EBM - ES resource, please, cite as follows:

@article{campillosetal-midm2021,
        title = {A clinical trials corpus annotated with UMLS© entities to enhance the access to Evidence - Based Medicine},
        author = {Campillos - Llanos, Leonardo and Valverde - Mateos, Ana and Capllonch - Carri{\'o}n, Adri{\'a}n and Moreno - Sandoval, Antonio},
        journal = {BMC Medical Informatics and Decision Making},
        volume={21},
        number={1},
        pages={1--19},
        year={2021},
        publisher={BioMed Central}
}

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e - 05
• train_batch_size: 16
• eval_batch_size: 16
• seed: we used different seeds for 5 evaluation rounds, and uploaded the model with the best results
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• num_epochs: average of 14 epochs (±2.24)

Training results (test set; average and standard deviation of 5 rounds with different seeds)

Results per class (test set; average and standard deviation of 5 rounds with different seeds)

Framework versions

📄 License

This model is licensed under the CC - BY - NC - 4.0 license.