Sd35m Reflow

🚀 sd35m-reflow

This project presents a standard PEFT LoRA derived from stabilityai/stable-diffusion-3.5-medium. It offers a solution for text - to - image and image - to - image tasks, enabling users to generate high - quality images with specific settings.

🚀 Quick Start

To start using this model, you can follow the inference code example provided below. It demonstrates how to load the base model, the adapter, and generate an image.

✨ Features

• Derived from a well - known base model stabilityai/stable-diffusion-3.5-medium.
• Offers specific validation and training settings for better control over image generation.
• Allows for inference with a provided Python code example.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

import torch
from diffusers import DiffusionPipeline

model_id = 'stabilityai/stable-diffusion-3.5-medium'
adapter_id = 'bghira/sd35m-reflow'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "A photo-realistic image of a cat"
negative_prompt = 'blurry, cropped, ugly'

## Optional: quantise the model to save on vram.
## Note: The model was not quantised during training, so it is not necessary to quantise it during inference time.
#from optimum.quanto import quantize, freeze, qint8
#quantize(pipeline.transformer, weights=qint8)
#freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=8,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=1.0,
    skip_guidance_layers=[],
).images[0]

model_output.save("output.png", format="PNG")

📚 Documentation

Validation settings

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Datasets - photo10k

🔧 Technical Details

This section provides detailed information about the validation, training settings, and datasets used in the project. The validation settings define the parameters for validating the model, while the training settings cover aspects such as learning rate, batch size, and optimizer. The dataset section describes the characteristics of the photo10k dataset used in the project.

📄 License

The license is specified as "other".