FLUX.1 Controlnet Lineart Promeai

🚀 promeai/FLUX.1-controlnet-lineart-promeai

promeai/FLUX.1-controlnet-lineart-promeai contains ControlNet weights trained on black-forest-labs/FLUX.1-dev with lineart conditions. It enables text-to-image generation with specific lineart guidance, offering unique and detailed image outputs.

🚀 Quick Start

Here are some example images generated by the model.

Prompt: cute anime girl with massive fluffy fennec ears and a big fluffy tail blonde messy long hair blue eyes wearing a maid outfit with a long black gold leaf pattern dress and a white apron mouth open holding a fancy black forest cake with candles on top in the kitchen of an old dark Victorian mansion lit by candlelight with a bright window to the foggy forest and very expensive stuff everywhere

Input Control	Result Image

✨ Features

• Text-to-Image Generation: Generate high - quality images based on text prompts with lineart control.
• ControlNet Integration: Utilize ControlNet to enhance the generation process and achieve more precise results.

📦 Installation

This project relies on the diffusers library. You can install it using the following command:

pip install diffusers

💻 Usage Examples

Basic Usage

With diffusers

import torch
from diffusers.utils import load_image
from diffusers.pipelines.flux.pipeline_flux_controlnet import FluxControlNetPipeline
from diffusers.models.controlnet_flux import FluxControlNetModel

base_model = 'black-forest-labs/FLUX.1-dev'
controlnet_model = 'promeai/FLUX.1-controlnet-lineart-promeai'
controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16)
pipe = FluxControlNetPipeline.from_pretrained(base_model, controlnet=controlnet, torch_dtype=torch.bfloat16)
pipe.to("cuda")

control_image = load_image("./images/example-control.jpg")
prompt = "cute anime girl with massive fluffy fennec ears and a big fluffy tail blonde messy long hair blue eyes wearing a maid outfit with a long black gold leaf pattern dress and a white apron mouth open holding a fancy black forest cake with candles on top in the kitchen of an old dark Victorian mansion lit by candlelight with a bright window to the foggy forest and very expensive stuff everywhere"
image = pipe(
    prompt, 
    control_image=control_image,
    controlnet_conditioning_scale=0.6,
    num_inference_steps=28, 
    guidance_scale=3.5,
).images[0]
image.save("./image.jpg")

With comfyui

An example comfyui workflow is also provided.

📚 Documentation

Limitations and bias

⚠️ Important Note

• This model is not intended or able to provide factual information.
• As a statistical model, this checkpoint might amplify existing societal biases.
• The model may fail to generate output that matches the prompts.
• Prompt following is heavily influenced by the prompting - style.

Training details

This ControlNet is trained on one A100 - 80G GPU, with a carefully selected proprietary real - world images dataset. The training process has two phases: an earlier period with an image size of 512 and a batch size of 3, and after the 512 - size training, an image size of 1024 and a batch size of 1. With these configurations, the GPU memory usage was about 70G, and it took around 3 days to reach the 14000 - steps checkpoint.

📄 License

No license information is provided in the original document.