# Self-supervised Learning

The first comprehensive benchmark study model for self-supervised learning on 3D medical imaging data, providing multiple pre-trained checkpoints

A self-supervised pre-trained foundation model for ECG analysis, supporting detection of 164 cardiovascular diseases

Path Foundation is a machine learning model for histopathology applications, trained through self-supervised learning to generate 384-dimensional embedding vectors from H&E-stained slides for efficient classifier model training.

A multilingual vision-language pre-trained model for the remote sensing field, supporting image-text cross-modal tasks in 10 languages.

DINOv2 is a visual feature extraction model based on Vision Transformer (ViT), which enhances feature extraction capabilities by introducing register mechanisms.

RAD-DINO-MAIRA-2 is a vision transformer model trained with DINOv2 self-supervised learning, specifically designed for encoding chest X-ray images.

DaSheng is a general-purpose audio encoder trained with large-scale self-supervised learning, capable of capturing rich audio information across multiple domains such as speech, music, and environmental sounds.

wav2vec 2.0 is a self-supervised learning framework for speech representation learning, capable of learning speech features from unlabeled audio data.

Phikon-v2 is a model based on the Vision Transformer Large architecture, pre-trained on the PANCAN-XL dataset using the DinoV2 self-supervised method, specifically designed for histological image analysis.

VQVAE is a video generation model based on the VQ-VAE architecture, cloned from the VideoGPT project, aimed at converting the model to Hugging Face format for easier loading.

Audio representation learning model based on HuBERT architecture, pre-trained on the complete AudioSet dataset

A Transformer model based on the HuBERT architecture, pre-trained on the complete AudioSet dataset, suitable for general audio representation learning tasks.

Audio representation model based on HuBERT architecture, pretrained on the complete AudioSet dataset, suitable for general audio tasks

TwinBooster is a DeBERTa V3 base model fine-tuned on the PubChem bioassay corpus, combined with the Barlow Twins self-supervised learning method for molecular property prediction.

Hubert (Hidden-Unit BERT) is a speech representation learning model proposed by Facebook, which uses self-supervised learning to directly learn speech features from raw waveform signals.

VideoMAE is a masked autoencoder model for video, pretrained with self-supervision and fine-tuned on the Kinetics-400 dataset, suitable for video classification tasks.

A Vision Transformer (ViT) image feature model trained with self-supervised DINO method, suitable for image classification and feature extraction tasks.

A ResNet-50 model pre-trained using the DINO self-supervised learning method, suitable for visual feature extraction tasks

KRISSBERT is a knowledge-enhanced self-supervised learning model for biomedical entity linking. It trains contextual encoders using unannotated text and domain knowledge to effectively address the diversity and ambiguity of entity names.

ProtGPT2 is a protein language model based on the GPT2 architecture, capable of generating novel protein sequences while retaining key features of natural proteins.

The first ALBERT model specifically for Persian, based on Google's ALBERT base v2.0 architecture, trained on diverse Persian corpora

A lightweight BERT model for self-supervised language representation learning in Persian

Lightweight BERT model designed for self-supervised learning of Persian language representations

A lightweight BERT model designed for Persian self-supervised language representation learning

Splinter is a self-supervised pre-trained model specifically designed for few-shot QA tasks, utilizing the Recurring Span Selection (RSS) objective for pre-training.

The Persian ALBERT model is a lightweight BERT for self-supervised learning of Persian language representations

A lightweight BERT model for self-supervised language representation learning in Persian

A lightweight BERT model for self-supervised learning of Persian language representations

DistilHuBERT is a lightweight speech representation learning model achieved through hierarchical distillation of the HuBERT model, significantly reducing model size and computational costs while maintaining performance.

Splinter is a few-shot QA model pre-trained via self-supervised learning, utilizing the Recurrent Span Selection (RSS) objective to simulate the span selection process in extractive QA.

A lightweight BERT model for self-supervised language representation learning in Persian

A wav2vec2 base model trained on 1K hours of French speech, supporting tasks like speech recognition

Large wav2vec2 model trained on 7.6K hours of French speech, including spontaneous, read, and broadcast speech

This is a Dutch speech recognition model based on the wav2vec2 architecture, developed by Facebook and specifically optimized for Dutch.

A lightweight BERT model for self-supervised language representation learning in Persian

Large wav2vec2 model trained on 7.6K hours of French speech data, including spontaneous, read, and broadcast speech