Hubert Ecg Small

🚀 HuBERT-ECG: A Self-Supervised Foundation Model for Broad and Scalable Cardiac Application

HuBERT-ECG is a self - supervised pre - trained foundation model for electrocardiogram (ECG) analysis. It can be fine - tuned for various downstream tasks and shows excellent performance across diverse scenarios.

Original code at https://github.com/Edoar-do/HuBERT-ECG

🚀 Quick Start

HuBERT-ECG is a self - supervised pre - trained model on a large and diverse dataset of 9.1 million 12 - lead ECGs. It can be easily fine - tuned for different downstream tasks by adding an output layer.

✨ Features

• Versatile Downstream Tasks: By simply adding an output layer, HuBERT-ECG can be fine - tuned for a wide range of downstream tasks, such as diagnosing diseases and predicting future cardiovascular events.
• High Performance: Across diverse real - world scenarios, it achieves AUROCs from 84.3% in low - data settings to 99% in large - scale setups. When detecting 164 overlapping conditions simultaneously, it delivers AUROCs above 90% and 95% for 140 and 94 diseases respectively, and predicts death events within a 2 - year follow - up with an AUROC of 93.4%.

📦 Installation

The code uses the transformers library. You can install it via pip install transformers if not already installed.

💻 Usage Examples

Basic Usage

from transformers import AutoModel
size = 'small' # any size from small, base, large
hubert_ecg = AutoModel.from_pretrained(f"Edoardo-BS/hubert-ecg-{size}", trust_remote_code=True)

📚 Documentation

Abstract

Deep learning models have shown remarkable performance in electrocardiogram (ECG) analysis, but their success has been constrained by the limited availability and size of ECG datasets, resulting in systems that are more task specialists than versatile generalists. In this work, we introduce HuBERT-ECG, a foundation ECG model pre - trained in a self - supervised manner on a large and diverse dataset of 9.1 million 12 - lead ECGs encompassing 164 cardiovascular conditions. By simply adding an output layer, HuBERT-ECG can be fine - tuned for a wide array of downstream tasks, from diagnosing diseases to predicting future cardiovascular events. Across diverse real - world scenarios, HuBERT-ECG achieves AUROCs from 84.3% in low - data settings to 99% in large - scale setups. When trained to detect 164 overlapping conditions simultaneously, our model delivers AUROCs above 90% and 95% for 140 and 94 diseases, respectively. HuBERT-ECG also predicts death events within a 2 - year follow - up with an AUROC of 93.4%. We release models and code.

Models

This repository contains the self - supervised pre - trained hubert-ecg - small.

📄 License

License: CC BY - NC 4.0

⚠️ Important Note

Don't forget to pre - process your data! Read the paper to know more about it

📚 Citation

If you use our models or find our work useful, please consider citing us:

doi: https://doi.org/10.1101/2024.11.14.24317328