ControlNet Inpaint Dreamer SDXL Open-Source Model - Free for Image Restoration and Expansion, Compatible with Stable Diffusion XL

Controlnet Inpaint Dreamer Sdxl

Developed by destitech

A ControlNet model specifically designed for image inpainting and expansion, compatible with the Stable Diffusion XL framework

Image Generation OtherOpen Source License:Openrail #SDXL Image Inpainting #AI Image Expansion #ControlNet Inpainting

Downloads 1,127

Release Time : 9/24/2023

Model Overview

This ControlNet model is specially trained for image inpainting and expansion tasks, capable of precisely guiding the image generation process through the control network. It is particularly suitable for handling images that require local modifications or expansions.

Model Features

Image Inpainting and Expansion

Optimized specifically for image inpainting and expansion tasks, enabling precise control over generated content

SDXL Framework Compatibility

Fully compatible with Stable Diffusion XL and its derivative models

Flexible Control

Specify areas to be processed using pure white markers, while the rest can remain unchanged or vary according to prompts

Model Capabilities

Image Inpainting

Image Expansion

Local Image Modification

Prompt-Based Image Generation

Use Cases

Digital Art Creation

Restoring Old Photos

Repair photos with damaged or missing parts

Fill in missing content while preserving the original style

Expanding Image Borders

Extend image dimensions or change composition ratios

Generate expanded content that matches the original style

Concept Design

Rapid Prototyping

Modify and expand existing designs

Quickly generate multiple design variants

🚀 Controlnet - Inpainting dreamer

This ControlNet is conditioned on Inpainting and Outpainting, designed to work with Stable Diffusion XL.

This ControlNet has been conditioned on Inpainting and Outpainting.

It is an early alpha version made by experimenting in order to learn more about controlnet.

You want to support this kind of work and the development of this model? Feel free to buy me a coffee!

It is designed to work with Stable Diffusion XL. It should work with any model based on it.

The image to inpaint or outpaint is to be used as input of the controlnet in a txt2img pipeline with denoising set to 1.0. The part to in/outpaint should be colors in solid white.

Depending on the prompts, the rest of the image might be kept as is or modified more or less.

✨ Features

Conditioned on Inpainting and Outpainting.
Compatible with Stable Diffusion XL and related models.

📚 Documentation

Model Details

Property	Details
Developed by	Destitech
Model Type	Controlnet
License	The CreativeML OpenRAIL M license is an Open RAIL M license, adapted from the work that BigScience and the RAIL Initiative are jointly carrying in the area of responsible AI licensing. See also the article about the BLOOM Open RAIL license on which our license is based.

Released Checkpoints

💻 Usage Examples

Usage with Diffusers

OzzyGT made a really good guide on how to use this model for outpainting, give it a try Here!

A big thank you to him for pointing me out how to name the files for diffusers compatibility and for the fp16 version, you should be able to use it this way with both normal and fp16 version:

from diffusers import ControlNetModel
import torch

controlnet = ControlNetModel.from_pretrained(
    "destitech/controlnet-inpaint-dreamer-sdxl", torch_dtype=torch.float16, variant="fp16"
)

Usage with ComfyUI

Workflow link

More examples

📄 License

This model is released under The CreativeML OpenRAIL M license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご