PhotoMaker Open-Source Text-to-Image Model - Generate Customized Works Quickly with Photos and Text without Training

Photomaker

Developed by TencentARC

PhotoMaker is a text-to-image generation model capable of quickly generating customized photos or artworks by inputting facial photos and text prompts, without requiring training.

Image Generation EnglishOpen Source License:Apache-2.0 #Zero-shot Customization #Multi-image Fusion #Cross-style Generation

Downloads 25.88k

Release Time : 1/13/2024

Model Overview

PhotoMaker allows users to obtain customized photos or artworks within seconds by inputting one or more facial photos along with text prompts. The model can be adapted to any SDXL-based foundation model or used in conjunction with other LoRA modules.

Model Features

No Training Required

Users can generate customized images by simply inputting facial photos and text prompts, without additional training.

Multi-style Adaptation

Can be adapted to any SDXL-based foundation model, supporting various styles including realistic and artistic.

Fast Generation

Generates high-quality customized images within seconds.

Multiple Photo Inputs

Supports input of one or more facial photos for personalized generation.

Model Capabilities

Text-to-image generation

Facial customization

Multi-style image generation

Fast inference

Use Cases

Personalized Image Generation

Realistic Style Portraits

Generate personalized portraits in realistic style by inputting facial photos and text prompts.

High-quality realistic style portrait images

Artistic Style Creations

Combine with artistic style foundation models to generate personalized images with artistic flair.

Diverse artistic style images

Creative Design

Character Design

Used for character design in fields such as games and animation.

Personalized character images

🚀 PhotoMaker Model Card

The PhotoMaker model allows users to input face photos and text prompts to quickly generate customized photos or paintings without the need for training. It can be adapted to SDXL - based base models and used with other LoRA modules.

🚀 Quick Start

You can directly download the model in this repository. You also can download the model in a Python script:

from huggingface_hub import hf_hub_download
photomaker_ckpt = hf_hub_download(repo_id="TencentARC/PhotoMaker", filename="photomaker-v1.bin", repo_type="model")

Then, please follow the instructions in our GitHub repository.

[**Project Page**](https://photo-maker.github.io/) **|** [**Paper (ArXiv)**](https://arxiv.org/abs/2312.04461) **|** [**Code**](https://github.com/TencentARC/PhotoMaker) [🤗 **Gradio demo (Realistic)**](https://huggingface.co/spaces/TencentARC/PhotoMaker) **|** [🤗 **Gradio demo (Stylization)**](https://huggingface.co/spaces/TencentARC/PhotoMaker-Style)

✨ Features

Users can input one or a few face photos, along with a text prompt, to receive a customized photo or painting within seconds (no training required!). Additionally, this model can be adapted to any base model based on SDXL or used in conjunction with other LoRA modules.

Realistic results

image/jpeg

Stylization results

image/jpeg

More results can be found in our project page

🔧 Technical Details

It mainly contains two parts corresponding to two keys in the loaded state dict:

id_encoder includes finetuned OpenCLIP - ViT - H - 14 and a few fuse layers.
lora_weights applies to all attention layers in the UNet, and the rank is set to 64.

📚 Documentation

Limitations

The model's customization performance degrades on Asian male faces.
The model still struggles with accurately rendering human hands.

Bias

While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.

Citation

BibTeX:

@inproceedings{li2023photomaker,
  title={PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding},
  author={Li, Zhen and Cao, Mingdeng and Wang, Xintao and Qi, Zhongang and Cheng, Ming - Ming and Shan, Ying},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}

📄 License

This model is licensed under the Apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご