đ Model Card for Puffin-Phi V2
Phi-1.5 fine tuned with Hermes Dataset, offering enhanced text generation capabilities.
đ Quick Start
Phi does not support device_map "auto", and does not seem to want to inference in fp16, so use bf16.
Here is working code to inference, though it can be improved:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("teknium/Puffin-Phi-v2", trust_remote_code=True, torch_dtype=torch.bfloat16).to("cuda")
tokenizer = AutoTokenizer.from_pretrained("teknium/Puffin-Phi-v2", trust_remote_code=True, torch_dtype=torch.bfloat16)
inputs = tokenizer(f"### Instruction:\nWrite a negative review for the website, Twitter.\n### Response:\n", return_tensors="pt", return_attention_mask=False)
outputs = model.generate(**inputs, max_length=128, do_sample=True, temperature=0.2, top_p=0.9, use_cache=True, repetition_penalty=1.2, eos_token_id=tokenizer.eos_token_id)
text = tokenizer.batch_decode(outputs)[0]
print(text)
The prompt format is Alpaca,
then is prompted like so:
### Instruction:
<prompt>
### Response:
⨠Features
This model was fine - tuned from Phi - 1.5 with the Hermes Dataset, leveraging a large number of synthetic datapoints.
đĻ Installation
No specific installation steps are provided in the original document.
đģ Usage Examples
Basic Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("teknium/Puffin-Phi-v2", trust_remote_code=True, torch_dtype=torch.bfloat16).to("cuda")
tokenizer = AutoTokenizer.from_pretrained("teknium/Puffin-Phi-v2", trust_remote_code=True, torch_dtype=torch.bfloat16)
inputs = tokenizer(f"### Instruction:\nWrite a negative review for the website, Twitter.\n### Response:\n", return_tensors="pt", return_attention_mask=False)
outputs = model.generate(**inputs, max_length=128, do_sample=True, temperature=0.2, top_p=0.9, use_cache=True, repetition_penalty=1.2, eos_token_id=tokenizer.eos_token_id)
text = tokenizer.batch_decode(outputs)[0]
print(text)
đ Documentation
Model Details
Model Sources
This model was trained on the OpenHermes Dataset, made by me, which is over 240,000 mostly GPT - 4 generated synthetic datapoints


Uses
Let me know!
Training Details
Training Procedure
Trained with Axolotl. View the wandb runs for all my puffin runs (this is puffin - phi - 4 on wandb):
https://wandb.ai/teknium1/hermes-phi/runs/hermes-phi-1
Evaluation

đ License
License: other
Property |
Details |
Pipeline Tag |
text-generation |
Datasets |
teknium/openhermes |