Hidream I1

🚀 hidream-controlnet-lora-test

This project presents a ControlNet PEFT LoRA derived from HiDream-ai/HiDream-I1-Full. It offers capabilities in text - to - image and image - to - image generation, leveraging the power of diffusers and LoRA techniques.

🚀 Quick Start

Prerequisites

Before using this project, ensure you have the necessary libraries installed. You can install them using pip or other package managers.

Inference

The following is an example of how to perform inference with this model:

import torch
from diffusers import DiffusionPipeline

model_id = 'HiDream-ai/HiDream-I1-Full'
adapter_id = 'bghira/hidream-controlnet-lora-test'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "A photo-realistic image of a cat"
negative_prompt = 'ugly, cropped, blurry, low-quality, mediocre average'

## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=16,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=256,
    height=256,
    guidance_scale=4.0,
).images[0]

model_output.save("output.png", format="PNG")

✨ Features

• Based on ControlNet PEFT LoRA: Derived from HiDream-ai/HiDream-I1-Full, it provides enhanced control over image generation.
• Multiple Generation Modes: Supports text - to - image and image - to - image generation.
• Flexible Configuration: Allows users to adjust various parameters during training and inference.

📦 Installation

The installation process mainly involves installing the necessary Python libraries. You can use the following command to install the required libraries:

pip install diffusers torch optimum

💻 Usage Examples

Basic Usage

The code in the Quick Start section demonstrates the basic usage of this model for text - to - image generation.

Advanced Usage

You can adjust the parameters such as prompt, negative_prompt, num_inference_steps, etc., to generate different images according to your needs.

📚 Documentation

Validation settings

• CFG: 4.0
• CFG Rescale: 0.0
• Steps: 16
• Sampler: FlowMatchEulerDiscreteScheduler
• Seed: 42
• Resolution: 256x256

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery: