PubMedBert-abstract-cord19 Open-source Model - For Biomedical Text Processing and Analysis

Home

Pubmedbert Abstract Cord19

Developed by pritamdeka

Biomedical text processing model fine-tuned on the CORD-19 abstract dataset based on PubMedBERT

Large Language Model

Transformers

Open Source License:MIT #Biomedical Text Processing #PubMedBERT Fine-tuning #COVID-19 Research

Downloads 16

Release Time : 3/2/2022

Model Overview

This model is a fine-tuned version of PubMedBERT on the CORD-19 abstract dataset, specializing in biomedical text processing tasks.

Model Features

Biomedical Domain Optimization

Specially optimized for biomedical literature, particularly suitable for processing medical research abstracts

Efficient Fine-tuning

Fine-tuned for 3 epochs on the CORD-19 dataset with a learning rate of 5e-05

Mixed Precision Training

Uses Native AMP mixed precision training technology to improve training efficiency

Model Capabilities

Biomedical Text Understanding

Scientific Abstract Analysis

Medical Literature Processing

Use Cases

Medical Research

COVID-19 Literature Analysis

Processing and analyzing COVID-19 related research abstracts

Biomedical Information Extraction

Extracting key information from medical research abstracts

🚀 PubMedBert-abstract-cord19

This model is a fine - tuned version of [microsoft/BiomedNLP - PubMedBERT - base - uncased - abstract - fulltext](https://huggingface.co/microsoft/BiomedNLP - PubMedBERT - base - uncased - abstract - fulltext) on the [pritamdeka/cord - 19 - abstract](https://huggingface.co/datasets/pritamdeka/cord - 19 - abstract) dataset, aiming to provide more accurate results in relevant tasks.

🚀 Quick Start

This model is a fine - tuned version of [microsoft/BiomedNLP - PubMedBERT - base - uncased - abstract - fulltext](https://huggingface.co/microsoft/BiomedNLP - PubMedBERT - base - uncased - abstract - fulltext) on the [pritamdeka/cord - 19 - abstract](https://huggingface.co/datasets/pritamdeka/cord - 19 - abstract) dataset. It achieves the following results on the evaluation set:

Loss: 1.3005

📚 Documentation

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

🔧 Technical Details

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e - 05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 10000
num_epochs: 3.0
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.3774	0.15	5000	1.3212
1.3937	0.29	10000	1.4059
1.6812	0.44	15000	1.6174
1.4712	0.59	20000	1.4383
1.4293	0.73	25000	1.4356
1.4155	0.88	30000	1.4283
1.3963	1.03	35000	1.4135
1.3718	1.18	40000	1.3948
1.369	1.32	45000	1.3961
1.354	1.47	50000	1.3788
1.3399	1.62	55000	1.3866
1.3289	1.76	60000	1.3630
1.3155	1.91	65000	1.3609
1.2976	2.06	70000	1.3489
1.2783	2.2	75000	1.3333
1.2696	2.35	80000	1.3260
1.2607	2.5	85000	1.3232
1.2547	2.64	90000	1.3034
1.2495	2.79	95000	1.3035
1.2404	2.94	100000	1.3029

Framework versions

Transformers 4.17.0.dev0
Pytorch 1.10.0+cu111
Datasets 1.18.3
Tokenizers 0.11.0

📄 License

This model is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご