Instantid

🚀 InstantID模型卡片

InstantID是一種全新的、無需調優的先進方法，僅需單張圖像即可實現身份保留生成，支持多種下游任務。

🚀 快速開始

你可以直接在本倉庫中下載模型，也可以通過Python腳本進行下載：

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="InstantX/InstantID", filename="ControlNetModel/config.json", local_dir="./checkpoints")
hf_hub_download(repo_id="InstantX/InstantID", filename="ControlNetModel/diffusion_pytorch_model.safetensors", local_dir="./checkpoints")
hf_hub_download(repo_id="InstantX/InstantID", filename="ip-adapter.bin", local_dir="./checkpoints")

對於人臉編碼器，你需要通過此鏈接手動下載到models/antelopev2目錄。

# !pip install opencv-python transformers accelerate insightface
import diffusers
from diffusers.utils import load_image
from diffusers.models import ControlNetModel

import cv2
import torch
import numpy as np
from PIL import Image

from insightface.app import FaceAnalysis
from pipeline_stable_diffusion_xl_instantid import StableDiffusionXLInstantIDPipeline, draw_kps

# prepare 'antelopev2' under ./models
app = FaceAnalysis(name='antelopev2', root='./', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

# prepare models under ./checkpoints
face_adapter = f'./checkpoints/ip-adapter.bin'
controlnet_path = f'./checkpoints/ControlNetModel'

# load IdentityNet
controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16)

pipe = StableDiffusionXLInstantIDPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, torch_dtype=torch.float16
)
pipe.cuda()

# load adapter
pipe.load_ip_adapter_instantid(face_adapter)

💻 使用示例

基礎用法

# load an image
image = load_image("your-example.jpg")

# prepare face emb
face_info = app.get(cv2.cvtColor(np.array(face_image), cv2.COLOR_RGB2BGR))
face_info = sorted(face_info, key=lambda x:(x['bbox'][2]-x['bbox'][0])*x['bbox'][3]-x['bbox'][1])[-1] # only use the maximum face
face_emb = face_info['embedding']
face_kps = draw_kps(face_image, face_info['kps'])

pipe.set_ip_adapter_scale(0.8)

prompt = "analog film photo of a man. faded film, desaturated, 35mm photo, grainy, vignette, vintage, Kodachrome, Lomography, stained, highly detailed, found footage, masterpiece, best quality"
negative_prompt = "(lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured (lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch,deformed, mutated, cross-eyed, ugly, disfigured"

# generate image
image = pipe(
    prompt, image_embeds=face_emb, image=face_kps, controlnet_conditioning_scale=0.8
).images[0]

高級用法

# 更多詳細用法，請參考我們的[GitHub倉庫](https://github.com/InstantID/InstantID)中的說明。

💡 使用建議

1. 如果你對生成圖像與原圖像的相似度不滿意，可以嘗試增加“IdentityNet強度”和“適配器強度”的權重。
2. 如果你覺得生成圖像的飽和度太高，可先降低適配器強度。若飽和度仍然過高，再降低IdentityNet強度。
3. 如果你發現文本控制效果未達預期，可降低適配器強度。
4. 如果你覺得生成圖像的寫實風格不夠好，可以前往我們的GitHub倉庫，使用更寫實的基礎模型。

✨ 演示示例

📄 許可證

本項目遵循Apache許可證發佈，旨在對人工智能驅動的圖像生成領域產生積極影響。用戶可以自由使用此工具創建圖像，但有義務遵守當地法律並負責任地使用。開發者不會對用戶的潛在不當使用行為承擔任何責任。

📚 引用信息

@article{wang2024instantid,
  title={InstantID: Zero-shot Identity-Preserving Generation in Seconds},
  author={Wang, Qixun and Bai, Xu and Wang, Haofan and Qin, Zekui and Chen, Anthony},
  journal={arXiv preprint arXiv:2401.07519},
  year={2024}
}

項目鏈接： 項目主頁 | 論文 | 代碼 | 🤗 Gradio演示