Open-source TwinBooster Model - Fine-tuned based on PubChem, Super Practical for Enhancing Molecular Property Prediction!

Pubchemdeberta Augmented

Developed by mschuh

TwinBooster is a DeBERTa V3 base model fine-tuned on the PubChem bioassay corpus, combining Barlow Twins self-supervised learning method and gradient boosting techniques to enhance molecular property prediction.

Molecular Model

Transformers

English#Zero-shot molecular prediction #Biological assay data augmentation #DeBERTa-v3 fine-tuning

Downloads 25

Release Time : 8/22/2023

Model Overview

This model integrates large language models with Barlow Twins and gradient boosting techniques to improve the prediction accuracy of molecular activity and properties, particularly suitable for early-stage active molecule identification in drug development.

Model Features

Zero-shot learning capability

Capable of predicting unknown biological assays and molecular properties, suitable for data-scarce scenarios.

Multimodal data fusion

Simultaneously utilizes assay information and molecular fingerprints to extract authentic molecular information.

Self-supervised learning

Employs the Barlow Twins method for self-supervised learning to enhance model performance.

Model Capabilities

Molecular property prediction

Biological assay data classification

Zero-shot learning

Use Cases

Drug development

Early-stage active molecule identification

Accelerates the identification process of active molecules in the early stages of drug development.

Demonstrated outstanding performance in the FS-Mol benchmark.

🚀 TwinBooster

TwinBooster is a fine - tuned DeBERTa V3 model on PubChem bioassay corpora, aiming to enhance molecular property prediction by synergizing large language models with Barlow Twins and gradient boosting.

🚀 Quick Start

There is no specific quick - start content provided in the original document.

✨ Features

Integrated Biological Assays: Utilizes a fine - tuned large language model to integrate biological assays based on textual information.
Self - supervised Learning: Couples with Barlow Twins, a Siamese neural network using a novel self - supervised learning approach.
Zero - shot Learning: Enables the prediction of properties of unseen bioassays and molecules through state - of - the - art zero - shot learning tasks.
High Performance: Shows excellent performance on the FS - Mol benchmark.

📚 Documentation

Abstract

The success of drug discovery and development relies on the precise prediction of molecular activities and properties. While in silico molecular property prediction has shown remarkable potential, its use has been limited so far to assays for which large amounts of data are available. In this study, we use a fine - tuned large language model to integrate biological assays based on their textual information, coupled with Barlow Twins, a Siamese neural network using a novel self - supervised learning approach. This architecture uses both assay information and molecular fingerprints to extract the true molecular information. TwinBooster enables the prediction of properties of unseen bioassays and molecules by providing state - of - the - art zero - shot learning tasks. Remarkably, our artificial intelligence pipeline shows excellent performance on the FS - Mol benchmark. This breakthrough demonstrates the application of deep learning to critical property prediction tasks where data is typically scarce. By accelerating the early identification of active molecules in drug discovery and development, this method has the potential to help streamline the identification of novel therapeutics.

Authors

Maximilian G. Schuh, Davide Boldini, Stephan A. Sieber

@ Chair of Organic Chemistry II, TUM School of Natural Sciences, Technical University of Munich

Paper Link

Metrics

perplexity

Pipeline Tag

sentence - similarity

Property	Details
Model Type	TwinBooster (PubChemDeBERTa - augmented, fine - tuned DeBERTa V3 base on PubChem bioassay corpora)
Training Data	PubChem bioassay corpora

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご