Photomaker V2

🚀 PhotoMaker V2 Model Card

Users can input face photos and text prompts to get customized photos or paintings in seconds. The model is adaptable to SDXL - based base models and can work with other LoRA modules.

🚀 Quick Start

You can directly download the model in this repository. You also can download the model in a Python script:

from huggingface_hub import hf_hub_download
photomaker_ckpt = hf_hub_download(repo_id="TencentARC/PhotoMaker-V2", filename="photomaker-v2.bin", repo_type="model")

Then, please follow the instructions in our GitHub repository.

✨ Features

Realistic results

Stylization results

More results can be found in our project page

🔧 Technical Details

It mainly contains two parts corresponding to two keys in the loaded state dict:

1. id_encoder includes finetuned OpenCLIP - ViT - H - 14 and a few fuse layers.

2. lora_weights applies to all attention layers in the UNet, and the rank is set to 64.

📚 Documentation

Limitations

• The model's customization performance degrades on Asian male faces.
• The model still struggles with accurately rendering human hands.

Bias

While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.

Citation

BibTeX:

@inproceedings{li2023photomaker,
  title={PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding},
  author={Li, Zhen and Cao, Mingdeng and Wang, Xintao and Qi, Zhongang and Cheng, Ming - Ming and Shan, Ying},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}

📄 License

This model is licensed under the Apache - 2.0 license.