Progen2 Small

🚀 ProGen2-small Model Mirror

This is a mirror of the base ProGen2-small model (with slightly modified configuration and forward pass) by Nijkamp, et al.. It provides an alternative way to access and utilize this model. You can also find an example of finetuning this model in my GitHub repo.

🚀 Quick Start

This project offers a mirror of the ProGen2-small model. You can use the following code to quickly start using the model.

💻 Usage Examples

Basic Usage

from transformers import AutoModelForCausalLM
from tokenizers import Tokenizer
import torch
import torch.nn.functional as F

# load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("hugohrban/progen2-small", trust_remote_code=True)
tokenizer = Tokenizer.from_pretrained("hugohrban/progen2-small")
tokenizer.no_padding()

# prepare input
prompt = "1MEVVIVTGMSGAGK"
input_ids = torch.tensor(tokenizer.encode(prompt).ids).to(model.device)

# forward pass
logits = model(input_ids).logits

# print output probabilities
next_token_logits = logits[-1, :]
next_token_probs = F.softmax(next_token_logits, dim=-1)
for i in range(tokenizer.get_vocab_size(with_added_tokens=False)):
    print(f"{tokenizer.id_to_token(i)}: {100 * next_token_probs[i].item():.2f} %")

📄 License

This project is licensed under the BSD 3-Clause license.