t2iadapter_openpose_sd14v1 Open-Source Model - Provide Pose Condition Control for Stable Diffusion 1.4

T2iadapter Openpose Sd14v1

Developed by TencentARC

T2I Adapter is a network that provides additional conditional control for stable diffusion models. This checkpoint is specifically designed to offer Openpose-based pose control for Stable Diffusion version 1.4.

Image Generation OtherOpen Source License:Apache-2.0 #Pose Control Generation #Skeleton Map Guidance #Precise Motion Reproduction

Downloads 35

Release Time : 7/14/2023

Model Overview

This model uses Openpose skeleton maps as conditional input to achieve precise control over the poses of images generated by the stable diffusion model, suitable for scenarios requiring specific poses.

Model Features

Precise Pose Control

Achieves precise control over human poses through Openpose skeleton maps.

Lightweight Adapter

Uses a lightweight adapter structure, enabling conditional control without modifying the base diffusion model.

Multi-Condition Support

Can be combined with text prompts to achieve image generation under multiple conditions.

Model Capabilities

Pose-Controlled Image Generation

Text-to-Image Conversion

Human Pose Preservation

Use Cases

Artistic Creation

Character Design

Generate character concept art with specific poses.

Precisely controls the actions and poses of generated characters.

Animation Production

Generate keyframes for animated characters with specific actions.

Maintains character pose consistency while enabling creative expression.

Virtual Avatars

Virtual Idols

Generate images of virtual idols performing specific dance moves.

Precisely controls the dance poses of virtual idols.

🚀 T2I Adapter - Openpose

T2I Adapter is a network that provides additional conditioning for Stable Diffusion. Each T2I checkpoint takes a different type of conditioning as input and is used with a specific base Stable Diffusion checkpoint. This checkpoint offers conditioning on OpenPose for the Stable Diffusion 1.4 checkpoint.

🚀 Quick Start

Dependencies

pip install diffusers transformers controlnet_aux

Run code

from PIL import Image
from diffusers import T2IAdapter, StableDiffusionAdapterPipeline
import torch
from controlnet_aux import OpenposeDetector

openpose = OpenposeDetector.from_pretrained('lllyasviel/ControlNet')

image = Image.open('./images/openpose_input.png')

image = openpose(image)

image.save('./images/openpose.png')

adapter = T2IAdapter.from_pretrained("TencentARC/t2iadapter_openpose_sd14v1", torch_dtype=torch.float16)
pipe = StableDiffusionAdapterPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4", adapter=adapter, safety_checker=None, torch_dtype=torch.float16, variant="fp16"
)

pipe.to('cuda')

generator = torch.Generator().manual_seed(1)

openpose_out = pipe(prompt="iron man flying", image=image, generator=generator).images[0]

openpose_out.save('./images/openpose_out.png')

openpose_input openpose_out

✨ Features

T2I Adapter enhances the controllability of Stable Diffusion by providing different types of conditioning inputs.

📚 Documentation

Model Details

Property	Details
Developed by	T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
Model Type	Diffusion-based text-to-image generation model
Language(s)	English
License	Apache 2.0
Resources for more information	GitHub Repository, Paper.
Cite as	@misc{ title={T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models}, author={Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, Xiaohu Qie}, year={2023}, eprint={2302.08453}, archivePrefix={arXiv}, primaryClass={cs.CV} }

Checkpoints

Model Name	Control Image Overview	Control Image Example	Generated Image Example
TencentARC/t2iadapter_color_sd14v1 Trained with spatial color palette	A image with 8x8 color palette.
TencentARC/t2iadapter_canny_sd14v1 Trained with canny edge detection	A monochrome image with white edges on a black background.
TencentARC/t2iadapter_sketch_sd14v1 Trained with PidiNet edge detection	A hand-drawn monochrome image with white outlines on a black background.
TencentARC/t2iadapter_depth_sd14v1 Trained with Midas depth estimation	A grayscale image with black representing deep areas and white representing shallow areas.
TencentARC/t2iadapter_openpose_sd14v1 Trained with OpenPose bone image	A OpenPose bone image.
TencentARC/t2iadapter_keypose_sd14v1 Trained with mmpose skeleton image	A mmpose skeleton image.
TencentARC/t2iadapter_seg_sd14v1 Trained with semantic segmentation	An custom segmentation protocol image.
TencentARC/t2iadapter_canny_sd15v2
TencentARC/t2iadapter_depth_sd15v2
TencentARC/t2iadapter_sketch_sd15v2
TencentARC/t2iadapter_zoedepth_sd15v1

📄 License

This project is licensed under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご