Flux.1-dev-Controlnet-Depth Open-source Image Conversion Model

Flux.1 Dev Controlnet Depth

Developed by jasperai

A depth map control network model developed by the Jasper research team for image-to-image tasks

Image Generation Open Source License:Other #Depth Map Controlled Generation #Artistic Image Synthesis #Multi-Condition Image Transformation

Downloads 1,499

Release Time : 9/17/2024

Model Overview

The Flux.1-dev Depth Control Network model is an image processing model based on the ControlNet architecture, primarily used for generating or transforming images based on depth map conditions.

Model Features

Depth Map Control

Can precisely control the geometric structure and spatial relationships of generated images based on input depth maps

High-Quality Image Generation

Generated images exhibit high visual quality and rich detail

Compatibility with Diffusers Library

Can be directly used with the diffusers library for easy integration into existing workflows

Model Capabilities

Depth map-based image generation

Image style transfer

Image content editing

Use Cases

Creative Design

Concept Art Creation

Generate artworks with specific spatial layouts based on depth maps

As shown in the example image of a gnome statue in a purple tulip field

Image Editing

Image Style Transfer

Change the style of an image while preserving its original spatial structure

🚀 Flux.1-dev: Depth ControlNet

This is a Depth map ControlNet for Flux.1-dev developed by the Jasper research team, which can be used in image - to - image tasks.

📦 Model Information

Property	Details
Base Model	black-forest-labs/FLUX.1-dev
Library Name	diffusers
License	other
License Name	flux-1-dev-non-commercial-license
License Link	https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
Pipeline Tag	image-to-image
Inference	False
Tags	ControlNet

🚀 Quick Start

This model can be used directly with the diffusers library.

💻 Usage Examples

Basic Usage

import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline

# Load pipeline
controlnet = FluxControlNetModel.from_pretrained(
  "jasperai/Flux.1-dev-Controlnet-Depth",
  torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
  "black-forest-labs/FLUX.1-dev",
  controlnet=controlnet,
  torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# Load a control image
control_image = load_image(
  "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/depth.jpg"
)

prompt = "a statue of a gnome in a field of purple tulips"

image = pipe(
    prompt, 
    control_image=control_image,
    controlnet_conditioning_scale=0.6,
    num_inference_steps=28, 
    guidance_scale=3.5,
    height=control_image.size[1],
    width=control_image.size[0]
).images[0]
image

Advanced Usage

💡 Note: You can compute the conditioning map using for instance the MidasDetector from the controlnet_aux library

from controlnet_aux import MidasDetector
from diffusers.utils import load_image 

midas = MidasDetector.from_pretrained("lllyasviel/Annotators")

midas.to("cuda")

# Load an image
im = load_image(
  "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/output.jpg"
)

depth = midas(im)

🔧 Technical Details

This model was trained with depth maps computed with Clipdrop's depth estimator model as well as open - source depth estimation models such as Midas or Leres.

📄 License

This model falls under the Flux.1-dev licence.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご