Open-source PubChemDeBERTa Model - Free Deployment to Boost Precise Prediction of Molecular Properties!

Pubchemdeberta

Developed by mschuh

TwinBooster is a DeBERTa V3 base model fine-tuned on the PubChem bioassay corpus, combined with the Barlow Twins self-supervised learning method for molecular property prediction.

Molecular Model

Transformers

English#Molecular Property Prediction #Zero-shot Learning #Bioassay Data Integration

Downloads 14

Release Time : 8/7/2023

Model Overview

This model integrates bioassay data with molecular fingerprint information to predict unknown bioassays and molecular properties, particularly suitable for molecular activity prediction in drug development.

Model Features

Barlow Twins Self-supervised Learning

Adopts the Barlow Twins twin neural network architecture to extract authentic molecular information through self-supervised learning.

Bioassay Data Integration

Combines textual information with bioassay data to enhance the accuracy of molecular property prediction.

Zero-shot Learning Capability

Demonstrates outstanding zero-shot learning capability in predicting unknown bioassays and molecular properties.

Model Capabilities

Molecular Property Prediction

Bioassay Data Analysis

Zero-shot Learning

Use Cases

Drug Development

Molecular Activity Prediction

Predicts molecular activity in specific bioassays, accelerating the identification of active molecules in early-stage drug development.

Demonstrates exceptional performance in the FS-Mol benchmark.

Cheminformatics

Molecular Fingerprint Extraction

Extracts molecular fingerprint information from bioassay data for cheminformatics analysis.

🚀 TwinBooster

Synergising Large Language Models with Barlow Twins and Gradient Boosting for Enhanced Molecular Property Prediction

🚀 Quick Start

This section is not provided in the original document.

✨ Features

Metric: Perplexity is used as a metric.
Pipeline Tag: It is designed for sentence - similarity tasks.
Fine - Tuned Model: PubChemDeBERTa is a fine - tuned DeBERTa V3 base on PubChem bioassay corpora.
Innovative Architecture: Integrates biological assays using a fine - tuned large language model and Barlow Twins, a Siamese neural network with a novel self - supervised learning approach.
Zero - Shot Learning: Enables the prediction of properties of unseen bioassays and molecules through state - of - the - art zero - shot learning tasks.
Benchmark Performance: Shows excellent performance on the FS - Mol benchmark.

📚 Documentation

Abstract

The success of drug discovery and development relies on the precise prediction of molecular activities and properties. While in silico molecular property prediction has shown remarkable potential, its use has been limited so far to assays for which large amounts of data are available. In this study, we use a fine - tuned large language model to integrate biological assays based on their textual information, coupled with Barlow Twins, a Siamese neural network using a novel self - supervised learning approach. This architecture uses both assay information and molecular fingerprints to extract the true molecular information. TwinBooster enables the prediction of properties of unseen bioassays and molecules by providing state - of - the - art zero - shot learning tasks. Remarkably, our artificial intelligence pipeline shows excellent performance on the FS - Mol benchmark. This breakthrough demonstrates the application of deep learning to critical property prediction tasks where data is typically scarce. By accelerating the early identification of active molecules in drug discovery and development, this method has the potential to help streamline the identification of novel therapeutics.

Paper Information

Authors: Maximilian G. Schuh, Davide Boldini, Stephan A. Sieber
Affiliation: Chair of Organic Chemistry II, TUM School of Natural Sciences, Technical University of Munich
arXiv Link:

Language

The project uses English.

Metrics

Perplexity is used as a metric.

Pipeline Tag

The pipeline tag is sentence - similarity.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご