medialbertina_pt-pt_900m: An Open-Source Medical Language Model Empowering Medical Applications with European Portuguese Data

Medialbertina Pt Pt 900m

Developed by portugueseNLP

The first publicly available medical language model trained on real European Portuguese data

OtherOpen Source License:MIT #Portuguese Medical NLP #Clinical Electronic Medical Record Analysis #European Portuguese Medical Model

Downloads 70

Release Time : 3/12/2024

Model Overview

A Bert-series encoder based on DeBERTaV2 architecture, specifically optimized for the European Portuguese medical domain, obtained through continued pre-training of the Albertina model with electronic medical record data

Model Features

Medical Domain Specialization

Trained on 2.6 million electronic medical records from Portugal's largest public hospital, specifically optimized for the medical domain

European Portuguese Support

Focuses on European Portuguese variants, especially language usage in medical scenarios

Performance Improvement

Outperforms the base model in named entity recognition and assertion status classification tasks with significant F1 score improvements

Large-scale Training Data

Trained with over 15 million sentences and 300 million tokens of real medical data

Model Capabilities

Medical Text Understanding

Clinical Entity Recognition

Medical Terminology Filling

Electronic Medical Record Processing

Use Cases

Clinical Information Extraction

Symptom Recognition

Identify symptom descriptions from clinical texts

F1 score 0.832 (NER task)

Medication Dosage Recognition

Identify medication and dosage information in prescriptions

Can accurately fill medication dosage units

Medical Text Completion

Clinical Record Completion

Automatically complete incomplete clinical records

Can accurately predict medical terms as shown in examples

🚀 MediAlbertina

The first publicly available medical language models trained with real European Portuguese data, offering enhanced performance for medical AI in Portugal.

🚀 Quick Start

MediAlbertina is a family of encoders from the Bert family, DeBERTaV2 - based. It results from the continuation of the pre - training of PORTULAN's Albertina models with Electronic Medical Records shared by Portugal's largest public hospital.

✨ Features

Domain Adaptation: MediAlbertina PT - PT 900M was created through domain adaptation of Albertina PT - PT 900M on real European Portuguese EMRs using masked language modeling.
Superior Performance: In Information Extraction (IE) tasks such as Named Entity Recognition (NER) and Assertion Status (AStatus), MediAlbertina achieved better F1 - scores compared to its predecessors, as shown in the table below:

Model	NER single - model	NER multi - models	Assertion Status
	F1 - score	F1 - score	F1 - score
albertina-900m-portuguese-ptpt-encoder	0.813	0.811	0.687
medialbertina_pt - pt_900m	0.832	0.848	0.755

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import pipeline

unmasker = pipeline('fill - mask', model='portugueseNLP/medialbertina_pt - pt_900m')
unmasker("Analgesia com morfina em perfusão (15 [MASK]/kg/h)")

Widget Examples

Here are some examples you can try:

Example Title	Input Text
Example 1	"Febre e tosse são sintomas comuns de [MASK]"
Example 2	"Diabetes [MASK] tipo II"
Example 3	"Utente tolera dieta [MASK] / Nivel de glicémia bom."
Example 4	"Doente com administração de [MASK] com tramal."
Example 5	"Colocada sonda de gases por apresentar [MASK] timpanizado"
Example 6	"Conectada em PRVC com necessidade de aumentar [MASK] para 70%"
Example 7	"Medicado com [MASK] em dias alternados."
Example 8	"Realizado teste de [MASK] ao paciente"
Example 9	"Sintomas apontam para COVID [MASK]."
Example 10	"Durante internamento fez [MASK] fresco congelado 3x dia"
Example 11	"Pupilas iso [MASK]."
Example 12	"Cardiopatia [MASK] - causa provável: HAS"
Example 13	"O paciente encontra - se [MASK] estável."
Example 14	"Traumatismo [MASK] após acidente de viação."
Example 15	"Analgesia com morfina em perfusão (15 [MASK]/kg/h)"

📚 Documentation

Data

MediAlbertina PT - PT 900M was trained on more than 15M sentences and 300M tokens from 2.6M fully anonymized and unique Electronic Medical Records (EMRs) from Portugal's largest public hospital. This data was acquired under the framework of the [FCT project DSAIPA/AI/0122/2020 AIMHealth - Mobile Applications Based on Artificial Intelligence](https://ciencia.iscte - iul.pt/projects/aplicacoes - moveis - baseadas - em - inteligencia - artificial - para - resposta - de - saude - publica/1567).

🔧 Technical Details

MediAlbertina models are distributed under the [MIT license](https://huggingface.co/portugueseNLP/medialbertina_pt - pt_900m/blob/main/LICENSE).

📄 License

MediAlbertina models are distributed under the [MIT license](https://huggingface.co/portugueseNLP/medialbertina_pt - pt_900m/blob/main/LICENSE).

Citation

MediAlbertina is developed by a joint team from [ISCTE - IUL](https://www.iscte - iul.pt/), Portugal, and Select Data, CA USA. For a fully detailed description, check the respective publication:

@article{MediAlbertina PT - PT,
      title={MediAlbertina: An European Portuguese medical language model}, 
      author={Miguel Nunes and João Boné and João Ferreira
              and Pedro Chaves and Luís Elvas},
      year={2024},
      journal={CBM},
      volume={182}
      url={https://doi.org/10.1016/j.compbiomed.2024.109233}
}

Please use the above cannonical reference when using or citing this model.

Acknowledgements

This work was financially supported by Project Blockchain.PT – Decentralize Portugal with Blockchain Agenda, (Project no 51), WP2, Call no 02/C05 - i01.01/2022, funded by the Portuguese Recovery and Resillience Program (PRR), The Portuguese Republic and The European Union (EU) under the framework of Next Generation EU Program.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご