Pickscore V1

🚀 PickScore v1模型卡片

PickScore v1是一個用於對文本生成圖像進行評分的模型。它以文本提示和生成的圖像作為輸入，輸出相應的評分。該模型可作為通用的評分函數，用於人類偏好預測、模型評估、圖像排序等任務。更多詳細信息請參考我們的論文 Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation。

🚀 快速開始

使用以下代碼開始使用該模型：

# import
from transformers import AutoProcessor, AutoModel

# load model
device = "cuda"
processor_name_or_path = "laion/CLIP-ViT-H-14-laion2B-s32B-b79K"
model_pretrained_name_or_path = "yuvalkirstain/PickScore_v1"

processor = AutoProcessor.from_pretrained(processor_name_or_path)
model = AutoModel.from_pretrained(model_pretrained_name_or_path).eval().to(device)

def calc_probs(prompt, images):
    
    # preprocess
    image_inputs = processor(
        images=images,
        padding=True,
        truncation=True,
        max_length=77,
        return_tensors="pt",
    ).to(device)
    
    text_inputs = processor(
        text=prompt,
        padding=True,
        truncation=True,
        max_length=77,
        return_tensors="pt",
    ).to(device)


    with torch.no_grad():
        # embed
        image_embs = model.get_image_features(**image_inputs)
        image_embs = image_embs / torch.norm(image_embs, dim=-1, keepdim=True)
    
        text_embs = model.get_text_features(**text_inputs)
        text_embs = text_embs / torch.norm(text_embs, dim=-1, keepdim=True)
    
        # score
        scores = model.logit_scale.exp() * (text_embs @ image_embs.T)[0]
        
        # get probabilities if you have multiple images to choose from
        probs = torch.softmax(scores, dim=-1)
    
    return probs.cpu().tolist()

pil_images = [Image.open("my_amazing_images/1.jpg"), Image.open("my_amazing_images/2.jpg")]
prompt = "fantastic, increadible prompt"
print(calc_probs(prompt, pil_images))

✨ 主要特性

該模型可作為通用的評分函數，用於人類偏好預測、模型評估、圖像排序等任務。

📚 詳細文檔

模型詳情

模型描述

該模型是基於CLIP - H，使用Pick - a - Pic數據集進行微調得到的。

模型來源

• 倉庫地址：查看PickScore倉庫
• 論文地址：Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
• 演示地址：PickScore的Huggingface Spaces演示

訓練詳情

訓練數據

該模型在Pick - a - Pic數據集上進行訓練。

訓練過程

待補充論文信息。

📄 許可證

文檔未提及相關許可證信息。

📖 引用

如果您覺得這項工作有用，請按以下格式引用：

@inproceedings{Kirstain2023PickaPicAO,
  title={Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation},
  author={Yuval Kirstain and Adam Polyak and Uriel Singer and Shahbuland Matiana and Joe Penna and Omer Levy},
  year={2023}
}

APA格式： [待補充更多信息]