ESM2 150M Protein Molecular Function

🚀 ESM2 Protein Function Caller

An Evolutionary-scale Model (ESM) for predicting protein functions from amino acid sequences using the Gene Ontology (GO). It offers insights into a protein's molecular function, biological process, and cellular location.

🚀 Quick Start

This Evolutionary-scale Model (ESM) predicts protein functions from amino acid sequences using the Gene Ontology (GO). Based on the ESM2 Transformer architecture, pre - trained on UniRef50 and fine - tuned on the AmiGO dataset, it predicts the GO subgraph for a specific protein sequence.

Note: This version only models the molecular function subgraph of the gene ontology.

✨ Features

• Library and Tags:
  • Library: transformers
  • Tags: gene - ontology, proteomics
• Datasets and Metrics:
  • Datasets: andrewdalpino/AmiGO
  • Metrics: precision, recall, f1
• Base Model and Pipeline:
  • Base Model: facebook/esm2_t30_150M_UR50D
  • Pipeline Tag: text - classification

📚 Documentation

What are GO terms?

"The Gene Ontology (GO) is a concept hierarchy that describes the biological function of genes and gene products at different levels of abstraction (Ashburner et al., 2000). It is a good model to describe the multi - faceted nature of protein function."

"GO is a directed acyclic graph. The nodes in this graph are functional descriptors (terms or classes) connected by relational ties between them (is_a, part_of, etc.). For example, terms 'protein binding activity' and 'binding activity' are related by an is_a relationship; however, the edge in the graph is often reversed to point from binding towards protein binding. This graph contains three subgraphs (subontologies): Molecular Function (MF), Biological Process (BP), and Cellular Component (CC), defined by their root nodes. Biologically, each subgraph represent a different aspect of the protein's function: what it does on a molecular level (MF), which biological processes it participates in (BP) and where in the cell it is located (CC)."

From [CAFA 5 Protein Function Prediction](https://www.kaggle.com/competitions/cafa - 5 - protein - function - prediction/data)

Code Repository

You can access the code repository at [https://github.com/andrewdalpino/esm2 - function - classifier](https://github.com/andrewdalpino/esm2 - function - classifier)

Model Specs

💻 Usage Examples

Basic Usage

For a basic demonstration, we can rank the GO terms for a particular sequence. For a more advanced example, see the [predict - subgraph.py](https://github.com/andrewdalpino/esm2 - function - classifier/blob/master/predict - subgraph.py) source file.

import torch

from transformers import EsmTokenizer, EsmForSequenceClassification

model_name = "andrewdalpino/ESM2-35M-Protein-Molecular-Function"

tokenizer = EsmTokenizer.from_pretrained(model_name)

model = EsmForSequenceClassification.from_pretrained(model_name)

model.eval()

sequence = "MCNAWYISVDFEKNREDKSKCIHTRRNSGPKLLEHVMYEVLRDWYCLEGENVYMM"

top_k = 10

out = tokenizer(sequence)

input_ids = out["input_ids"]

input_ids = torch.tensor(input_ids, dtype=torch.int64).unsqueeze(0)

with torch.no_grad():
    outputs = model.forward(input_ids)

    probabilities = torch.sigmoid(outputs.logits.squeeze(0))

    probabilities, indices = torch.topk(probabilities, top_k)

probabilities = probabilities.tolist()

terms = [model.config.id2label[index] for index in indices.tolist()]

print(f"Top {args.top_k} GO Terms:")

for term, probability in zip(terms, probabilities):
    print(f"{probability:.4f}: {term}")

📄 References

• A. Rives, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences, 2021.
• Z. Lin, et al. Evolutionary - scale prediction of atomic level protein structure with a language model, 2022.
• G. A. Merino, et al. Hierarchical deep learning for predicting GO annotations by integrating protein knowledge, 2022.
• M. Ashburner, et al. Gene Ontology: tool for the unification of biology, 2000.