OntoProtein Open-Source Protein Pre-Training Model - Optimizing Protein Sequence Representation through Knowledge Fusion

Ontoprotein

Developed by zjunlp

A protein pretraining model incorporating structured knowledge from Gene Ontology (GO), optimizing protein sequence representation through masked language modeling and knowledge embedding dual objectives.

Protein Model

Transformers

#Gene Ontology Fusion #Protein Knowledge Graph #Dual-Objective Pretraining

Downloads 69

Release Time : 3/2/2022

Model Overview

The first general framework integrating Gene Ontology knowledge into protein pretraining, achieving joint embedding learning of proteins and GO terms through large-scale knowledge graph construction.

Model Features

Knowledge-Enhanced Pretraining

Innovatively integrates structured Gene Ontology knowledge, optimizing protein representation through knowledge graph negative sampling contrastive learning

Dual-Objective Optimization

Simultaneously performs masked language modeling (MLM) for protein sequences and joint training of knowledge graph embedding (KE)

Large-Scale Knowledge Graph

Constructs a novel knowledge graph containing GO terms and associated proteins, with all nodes described by text or sequences

Model Capabilities

Protein sequence representation learning

Gene function prediction

Protein-knowledge graph joint embedding

Use Cases

Biomedical Research

Protein Function Annotation

Predicts unknown protein functions using GO knowledge-enhanced protein representations

Improves functional prediction accuracy compared to traditional methods

Protein-Protein Interaction Prediction

Computes protein similarity through knowledge-aware embedding space

🚀 OntoProtein model

A pre - trained model on protein sequences using masked language modeling (MLM) and knowledge embedding (KE) objectives, which leverages the structure in GO for protein pre - training.

🚀 Quick Start

The OntoProtein model is a pre - trained model on protein sequences. It uses masked language modeling (MLM) and knowledge embedding (KE) objectives. It was introduced in this paper and first released in this repository. Note that this model is trained on uppercase amino acids, so it only works with capital letter amino acids.

✨ Features

OntoProtein is the first general framework that incorporates the structure in GO (Gene Ontology) into protein pre - training models.
A novel large - scale knowledge graph consisting of GO and its related proteins is constructed. All nodes in the graph are described by gene annotation texts or protein sequences.
A novel contrastive learning method with knowledge - aware negative sampling is proposed to jointly optimize the knowledge graph and protein embedding during pre - training.

📚 Documentation

Model description

OntoProtein is the first general framework that makes use of structure in GO (Gene Ontology) into protein pre - training models. We construct a novel large - scale knowledge graph that consists of GO and its related proteins, and gene annotation texts or protein sequences describe all nodes in the graph. We propose novel contrastive learning with knowledge - aware negative sampling to jointly optimize the knowledge graph and protein embedding during pre - training.

BibTeX entry and citation info

@inproceedings{
zhang2022ontoprotein,
title={OntoProtein: Protein Pretraining With Gene Ontology Embedding},
author={Ningyu Zhang and Zhen Bi and Xiaozhuan Liang and Siyuan Cheng and Haosen Hong and Shumin Deng and Qiang Zhang and Jiazhang Lian and Huajun Chen},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/forum?id=yfe1VMYAXa4}
}

Additional Information

Property	Details
Model Type	Pretrained model on protein sequences using MLM and KE objectives
Training Data	Uppercase amino acids
Tags	protein language model
Datasets	ProteinKG25
Widget Example	"D L I P T S S K L V V [MASK] D T S L Q V K K A F F A L V T"

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご