HIV V3 Bodysite

🚀 Model Card for [HIV_V3_bodysite]

The HIV-BERT-Bodysite-Identification model is refined from the HIV-BERT model to better predict the origin location of an HIV V3 loop sample.

🚀 Quick Start

The HIV-BERT-Bodysite-Identification model can predict the likely body site from a V3 sequence. This may be used for surveillance of cells emerging from latent reservoirs. Remember, a sequence can come from multiple sites, and they are not mutually exclusive.

from transformers import pipeline

predictor = pipeline("text-classification", model="damlab/HIV_V3_bodysite")
predictor(f"C T R P N N N T R K S I R I Q R G P G R A F V T I G K I G N M R Q A H C")

[
  [
    {
      "label": "periphery-tcell",
      "score": 0.29097115993499756
    },
    {
      "label": "periphery-monocyte",
      "score": 0.014322502538561821
    },
    {
      "label": "CNS",
      "score": 0.06870711594820023
    },
    {
      "label": "breast-milk",
      "score": 0.002785981632769108
    },
    {
      "label": "female-genitals",
      "score": 0.024997007101774216
    },
    {
      "label": "male-genitals",
      "score": 0.01040483545511961
    },
    {
      "label": "gastric",
      "score": 0.06872137635946274
    },
    {
      "label": "lung",
      "score": 0.04432062804698944
    },
    {
      "label": "organ",
      "score": 0.47476938366889954
    }
  ]
]

✨ Features

• The model is a refinement of the HIV-BERT model, aiming to better predict the location from which an HIV V3 loop sample was derived.
• It can predict the likely body site from a V3 sequence, which is useful for cell surveillance from latent reservoirs.

📦 Installation

The README doesn't provide specific installation steps, so this section is skipped.

📚 Documentation

Summary

The HIV-BERT-Bodysite-Identification model was trained as a refinement of the HIV-BERT model (insert link) and serves to better predict the location that an HIV V3 loop sample was derived from. HIV-BERT is a model refined from the ProtBert-BFD model (https://huggingface.co/Rostlab/prot_bert_bfd) to better fulfill HIV-centric tasks. This model was then trained using HIV V3 sequences from the Los Alamos HIV Sequence Database (https://www.hiv.lanl.gov/content/sequence/HIV/mainpage.html), allowing even more precise prediction of body site location than the HIV-BERT model can provide.

Model Description

The HIV-BERT-Bodysite-Identification model is intended to predict the location as to where an HIV sequence was most likely derived from. Because HIV infects immune cells, it uses these as a means of rapidly spreading throughout the body. Thus, body site identification can help determine where exactly these HIV particles ultimately end up. This would be helpful when attempting to study HIV treatment strategies. When provided with an HIV genomic sequence, the HIV-BERT-Bodysite-Identification model can predict which tissue it was derived from.

Intended Uses & Limitations

This tool can be used as a predictor of which body site an HIV sample was derived from based on its genomic sequence. It should not be considered a clinical diagnostic tool.

This tool was trained using the Los Alamos HIV sequence dataset (https://www.hiv.lanl.gov/content/sequence/HIV/mainpage.html). Due to the sampling nature of this database, it is predominantly composed of subtype B sequences from North America and Europe with only minor contributions of Subtype C, A, and D. Currently, there was no effort made to balance the performance across these classes. As such, one should consider refinement with additional sequences to perform well on non-B sequences.

Training Data

This model was trained using the damlab/HIV_V3_bodysite dataset using the 0th fold. The dataset consists of 5510 sequences (approximately 35 tokens each) extracted from the Los Alamos HIV Sequence database.

Training Procedure

Preprocessing

As with the rostlab/Prot-bert-bfd model, the rare amino acids U, Z, O, and B were converted to X and spaces were added between each amino acid. All strings were concatenated and chunked into 256 token chunks for training. A random 20% of chunks were held for validation.

Training

The damlab/HIV-BERT model was used as the initial weights for an AutoModelforClassificiation. The model was trained with a learning rate of 1E-5, 50K warm-up steps, and a cosine_with_restarts learning rate schedule and continued until 3 consecutive epochs did not improve the loss on the held-out dataset. As this is a multiple classification task (a protein can be found in multiple sites) the loss was calculated as the Binary Cross Entropy for each category. The BCE was weighted by the inverse of the class ratio to balance the weight across the class imbalance.

Evaluation Results

The README indicates that evaluation results need to be added, so this section is skipped.

BibTeX Entry and Citation Info

The README states that more information is needed, so this section is skipped.

🔧 Technical Details

Preprocessing

As with the rostlab/Prot-bert-bfd model, the rare amino acids U, Z, O, and B were converted to X and spaces were added between each amino acid. All strings were concatenated and chunked into 256 token chunks for training. A random 20% of chunks were held for validation.

Training

The damlab/HIV-BERT model was used as the initial weights for an AutoModelforClassificiation. The model was trained with a learning rate of 1E-5, 50K warm-up steps, and a cosine_with_restarts learning rate schedule and continued until 3 consecutive epochs did not improve the loss on the held-out dataset. As this is a multiple classification task (a protein can be found in multiple sites) the loss was calculated as the Binary Cross Entropy for each category. The BCE was weighted by the inverse of the class ratio to balance the weight across the class imbalance.

📄 License

The model is released under the MIT license.