Open-source color adjustment tool - t2iadapter_color_sd14v1: Provides color control for Stable Diffusion version 1.4

T2iadapter Color Sd14v1

Developed by TencentARC

T2I Adapter is a network that provides additional conditional control for stable diffusion models. This checkpoint specifically offers palette-based color control for Stable Diffusion 1.4.

Image Generation OtherOpen Source License:Apache-2.0 #Palette Control #Color-guided Generation #SD1.4 Adaptation

Downloads 50

Release Time : 7/14/2023

Model Overview

T2I Adapter is a network designed to provide additional conditional control for stable diffusion models. This checkpoint is specialized for palette-based color control in Stable Diffusion 1.4.

Model Features

Color Control

Provides precise color control via an 8x8 palette, ensuring the generated image's color distribution matches the palette.

Lightweight Adapter

As a lightweight adapter network, it adds control functionality without modifying the original stable diffusion model.

Multi-condition Support

Supports various control conditions (color, edges, depth, etc.) and can be used with different base models.

Model Capabilities

Palette-based Image Generation

Color-controlled Image Synthesis

Conditional Image Generation

Use Cases

Artistic Creation

Color Stylization

Generates stylistically consistent images based on a specific palette.

The generated image's color distribution perfectly matches the input palette.

Design Assistance

Brand Visual Design

Generates visual elements based on corporate brand colors.

Ensures generated images comply with brand color guidelines.

🚀 T2I Adapter - Color

T2I Adapter is a network that provides additional conditioning to stable diffusion. It enables more controllable text - to - image generation.

🚀 Quick Start

Install Dependencies

pip install diffusers transformers

Run the Code

from PIL import Image
import torch
from diffusers import StableDiffusionAdapterPipeline, T2IAdapter

image = Image.open('./images/color_ref.png')

color_palette = image.resize((8, 8))
color_palette = color_palette.resize((512, 512), resample=Image.Resampling.NEAREST)

color_palette.save('./images/color_palette.png')

adapter = T2IAdapter.from_pretrained("TencentARC/t2iadapter_color_sd14v1", torch_dtype=torch.float16)
pipe = StableDiffusionAdapterPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4",
    adapter=adapter,
    torch_dtype=torch.float16,
)
pipe.to("cuda")

generator = torch.manual_seed(0)

out_image = pipe(
    "At night, glowing cubes in front of the beach",
    image=color_palette,
    generator=generator,
).images[0]

out_image.save('./images/color_out_image.png')

color_ref color_palette color_out_image

✨ Features

T2I Adapter is a network providing additional conditioning to stable diffusion. Each t2i checkpoint takes a different type of conditioning as input and is used with a specific base stable diffusion checkpoint. This checkpoint provides conditioning on color palettes for the stable diffusion 1.4 checkpoint.

📚 Documentation

Model Details

Property	Details
Developed by	T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
Model Type	Diffusion-based text-to-image generation model
Language(s)	English
License	Apache 2.0
Resources for more information	GitHub Repository, Paper.
Cite as	@misc{ title={T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models}, author={Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, Xiaohu Qie}, year={2023}, eprint={2302.08453}, archivePrefix={arXiv}, primaryClass={cs.CV} }

Checkpoints

Model Name	Control Image Overview	Control Image Example	Generated Image Example
TencentARC/t2iadapter_color_sd14v1 Trained with spatial color palette	A image with 8x8 color palette.
TencentARC/t2iadapter_canny_sd14v1 Trained with canny edge detection	A monochrome image with white edges on a black background.
TencentARC/t2iadapter_sketch_sd14v1 Trained with PidiNet edge detection	A hand-drawn monochrome image with white outlines on a black background.
TencentARC/t2iadapter_depth_sd14v1 Trained with Midas depth estimation	A grayscale image with black representing deep areas and white representing shallow areas.
TencentARC/t2iadapter_openpose_sd14v1 Trained with OpenPose bone image	A OpenPose bone image.
TencentARC/t2iadapter_keypose_sd14v1 Trained with mmpose skeleton image	A mmpose skeleton image.
TencentARC/t2iadapter_seg_sd14v1 Trained with semantic segmentation	An custom segmentation protocol image.
TencentARC/t2iadapter_canny_sd15v2
TencentARC/t2iadapter_depth_sd15v2
TencentARC/t2iadapter_sketch_sd15v2
TencentARC/t2iadapter_zoedepth_sd15v1

📄 License

This project is licensed under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご