ControlNet Inpaint Dreamer SDXL Open-Source Image Restoration Extension Model - Compatible with Stable Diffusion XL Framework

Controlnet Inpaint Dreamer Sdxl

Developed by frankjoshua

ControlNet model optimized for image inpainting and expansion, compatible with Stable Diffusion XL framework

Image Generation OtherOpen Source License:Openrail #SDXL Image Inpainting #AI Image Expansion #ControlNet Technology

Downloads 53

Release Time : 8/4/2024

Model Overview

This ControlNet is specifically designed for image inpainting and expansion tasks, an early alpha version developed through experiments, requiring use with Stable Diffusion XL. Processing areas must be marked as pure white with denoising strength set to 1.0.

Model Features

Image Inpainting & Expansion

Specially optimized for image inpainting and expansion tasks, capable of generating content for marked areas

Stable Diffusion XL Compatible

Fully compatible with Stable Diffusion XL and its derivative models, providing flexible generation options

Experimental ControlNet Technology

Utilizes cutting-edge ControlNet technology to explore possibilities of conditional image generation

Model Capabilities

Image inpainting

Image expansion

Conditional image generation

Use Cases

Digital Art Creation

Old Photo Restoration

Restoring damaged or missing parts of photographs

Filling missing content while preserving original style

Artwork Expansion

Extending the border areas of artworks

Seamless expansion maintaining style consistency

Concept Design

Scene Expansion

Adding environmental details to concept designs

Generating harmonious expanded content based on prompts

🚀 Controlnet - Inpainting dreamer

This ControlNet is conditioned on Inpainting and Outpainting, designed to work with Stable Diffusion XL.

🚀 Quick Start

This ControlNet has been conditioned on Inpainting and Outpainting. It is an early alpha version made by experimenting to learn more about controlnet.

You want to support this kind of work and the development of this model? Feel free to buy me a coffee!

It is designed to work with Stable Diffusion XL. It should work with any model based on it.

The image to inpaint or outpaint is to be used as input of the controlnet in a txt2img pipeline with denoising set to 1.0. The part to in/outpaint should be colored in solid white. Depending on the prompts, the rest of the image might be kept as is or modified more or less.

✨ Features

Conditioned on Inpainting and Outpainting.
Compatible with Stable Diffusion XL and models based on it.

📚 Documentation

Model Details

Property	Details
Developed by	Destitech
Model Type	Controlnet
License	The CreativeML OpenRAIL M license is an Open RAIL M license, adapted from the work that BigScience and the RAIL Initiative are jointly carrying in the area of responsible AI licensing. See also the article about the BLOOM Open RAIL license on which our license is based.

Released Checkpoints

💻 Usage Examples

Usage with Diffusers

OzzyGT made a really good guide on how to use this model for outpainting, give it a try Here!

A big thank you to him for pointing out how to name the files for diffusers compatibility and for the fp16 version. You should be able to use it this way with both normal and fp16 versions:

from diffusers import ControlNetModel
import torch

controlnet = ControlNetModel.from_pretrained(
    "destitech/controlnet-inpaint-dreamer-sdxl", torch_dtype=torch.float16, variant="fp16"
)

Usage with ComfyUI

Workflow link

More examples

📄 License

This model is under The CreativeML OpenRAIL M license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご