🚀 PickScore v1模型卡片
PickScore v1是一个用于对文本生成图像进行评分的模型。它以文本提示和生成的图像作为输入,输出相应的评分。该模型可作为通用的评分函数,用于人类偏好预测、模型评估、图像排序等任务。更多详细信息请参考我们的论文 Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation。
🚀 快速开始
使用以下代码开始使用该模型:
from transformers import AutoProcessor, AutoModel
device = "cuda"
processor_name_or_path = "laion/CLIP-ViT-H-14-laion2B-s32B-b79K"
model_pretrained_name_or_path = "yuvalkirstain/PickScore_v1"
processor = AutoProcessor.from_pretrained(processor_name_or_path)
model = AutoModel.from_pretrained(model_pretrained_name_or_path).eval().to(device)
def calc_probs(prompt, images):
image_inputs = processor(
images=images,
padding=True,
truncation=True,
max_length=77,
return_tensors="pt",
).to(device)
text_inputs = processor(
text=prompt,
padding=True,
truncation=True,
max_length=77,
return_tensors="pt",
).to(device)
with torch.no_grad():
image_embs = model.get_image_features(**image_inputs)
image_embs = image_embs / torch.norm(image_embs, dim=-1, keepdim=True)
text_embs = model.get_text_features(**text_inputs)
text_embs = text_embs / torch.norm(text_embs, dim=-1, keepdim=True)
scores = model.logit_scale.exp() * (text_embs @ image_embs.T)[0]
probs = torch.softmax(scores, dim=-1)
return probs.cpu().tolist()
pil_images = [Image.open("my_amazing_images/1.jpg"), Image.open("my_amazing_images/2.jpg")]
prompt = "fantastic, increadible prompt"
print(calc_probs(prompt, pil_images))
✨ 主要特性
该模型可作为通用的评分函数,用于人类偏好预测、模型评估、图像排序等任务。
📚 详细文档
模型详情
模型描述
该模型是基于CLIP - H,使用Pick - a - Pic数据集进行微调得到的。
模型来源
训练详情
训练数据
该模型在Pick - a - Pic数据集上进行训练。
训练过程
待补充论文信息。
📄 许可证
文档未提及相关许可证信息。
📖 引用
如果您觉得这项工作有用,请按以下格式引用:
@inproceedings{Kirstain2023PickaPicAO,
title={Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation},
author={Yuval Kirstain and Adam Polyak and Uriel Singer and Shahbuland Matiana and Joe Penna and Omer Levy},
year={2023}
}
APA格式:
[待补充更多信息]