Instantid

🚀 InstantID模型卡片

InstantID是一种全新的、无需调优的先进方法，仅需单张图像即可实现身份保留生成，支持多种下游任务。

🚀 快速开始

你可以直接在本仓库中下载模型，也可以通过Python脚本进行下载：

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="InstantX/InstantID", filename="ControlNetModel/config.json", local_dir="./checkpoints")
hf_hub_download(repo_id="InstantX/InstantID", filename="ControlNetModel/diffusion_pytorch_model.safetensors", local_dir="./checkpoints")
hf_hub_download(repo_id="InstantX/InstantID", filename="ip-adapter.bin", local_dir="./checkpoints")

对于人脸编码器，你需要通过此链接手动下载到models/antelopev2目录。

# !pip install opencv-python transformers accelerate insightface
import diffusers
from diffusers.utils import load_image
from diffusers.models import ControlNetModel

import cv2
import torch
import numpy as np
from PIL import Image

from insightface.app import FaceAnalysis
from pipeline_stable_diffusion_xl_instantid import StableDiffusionXLInstantIDPipeline, draw_kps

# prepare 'antelopev2' under ./models
app = FaceAnalysis(name='antelopev2', root='./', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

# prepare models under ./checkpoints
face_adapter = f'./checkpoints/ip-adapter.bin'
controlnet_path = f'./checkpoints/ControlNetModel'

# load IdentityNet
controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16)

pipe = StableDiffusionXLInstantIDPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, torch_dtype=torch.float16
)
pipe.cuda()

# load adapter
pipe.load_ip_adapter_instantid(face_adapter)

💻 使用示例

基础用法

# load an image
image = load_image("your-example.jpg")

# prepare face emb
face_info = app.get(cv2.cvtColor(np.array(face_image), cv2.COLOR_RGB2BGR))
face_info = sorted(face_info, key=lambda x:(x['bbox'][2]-x['bbox'][0])*x['bbox'][3]-x['bbox'][1])[-1] # only use the maximum face
face_emb = face_info['embedding']
face_kps = draw_kps(face_image, face_info['kps'])

pipe.set_ip_adapter_scale(0.8)

prompt = "analog film photo of a man. faded film, desaturated, 35mm photo, grainy, vignette, vintage, Kodachrome, Lomography, stained, highly detailed, found footage, masterpiece, best quality"
negative_prompt = "(lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured (lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch,deformed, mutated, cross-eyed, ugly, disfigured"

# generate image
image = pipe(
    prompt, image_embeds=face_emb, image=face_kps, controlnet_conditioning_scale=0.8
).images[0]

高级用法

# 更多详细用法，请参考我们的[GitHub仓库](https://github.com/InstantID/InstantID)中的说明。

💡 使用建议

1. 如果你对生成图像与原图像的相似度不满意，可以尝试增加“IdentityNet强度”和“适配器强度”的权重。
2. 如果你觉得生成图像的饱和度太高，可先降低适配器强度。若饱和度仍然过高，再降低IdentityNet强度。
3. 如果你发现文本控制效果未达预期，可降低适配器强度。
4. 如果你觉得生成图像的写实风格不够好，可以前往我们的GitHub仓库，使用更写实的基础模型。

✨ 演示示例

📄 许可证

本项目遵循Apache许可证发布，旨在对人工智能驱动的图像生成领域产生积极影响。用户可以自由使用此工具创建图像，但有义务遵守当地法律并负责任地使用。开发者不会对用户的潜在不当使用行为承担任何责任。

📚 引用信息

@article{wang2024instantid,
  title={InstantID: Zero-shot Identity-Preserving Generation in Seconds},
  author={Wang, Qixun and Bai, Xu and Wang, Haofan and Qin, Zekui and Chen, Anthony},
  journal={arXiv preprint arXiv:2401.07519},
  year={2024}
}

项目链接： 项目主页 | 论文 | 代码 | 🤗 Gradio演示