Sdxs 512 0.9

🚀 SDXS-512-0.9

SDXS is a real - time high - resolution image generation model based on prompt texts, trained using score distillation and feature matching.

🚀 Quick Start

The latest community version SDXS-512-DreamShaper is now available. It offers better quality and faster performance.

✨ Features

SDXS can generate high - resolution images in real - time from prompt texts. It is trained using score distillation and feature matching. For more details, check out our research paper: SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions.

SDXS-512-0.9 is an old version of SDXS-512. To avoid potential commercial and copyright risks, SDXS-512-1.0 and SDXS-1024-1.0 will soon be unavailable. Instead, we'll release new versions with different teacher DM or offline DM. Keep an eye on our repo for updates.

Model Information

Main Differences from Version 1.0

1. This version uses TAESD, which may generate low - quality images when weight_type is float16. Our image decoder is incompatible with the current diffusers version, so it won't be provided now.
2. This version didn't perform the LoRA - GAN finetune mentioned in the implementation details, resulting in slightly less detailed images.
3. This version replaces self - attention with cross - attention in the highest resolution stages, introducing minimal overhead compared to direct removal.

💻 Usage Examples

Basic Usage

import torch
from diffusers import StableDiffusionPipeline, AutoencoderKL

repo = "IDKiro/sdxs-512-0.9"
seed = 42
weight_type = torch.float32     # or float16

# Load model.
pipe = StableDiffusionPipeline.from_pretrained(repo, torch_dtype=weight_type)

# use original VAE
# pipe.vae = AutoencoderKL.from_pretrained("IDKiro/sdxs-512-0.9/vae_large")
 
pipe.to("cuda")

prompt = "portrait photo of a girl, photograph, highly detailed face, depth of field, moody light, golden hour"

# Ensure using 1 inference step and CFG set to 0.
image = pipe(
    prompt, 
    num_inference_steps=1, 
    guidance_scale=0,
    generator=torch.Generator(device="cuda").manual_seed(seed)
).images[0]

image.save("output.png")

📄 License

The model is released under the OpenRAIL++ license.

📚 Documentation

For more information about SDXS, please refer to our research paper: SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions.

📚 Cite Our Work

@article{song2024sdxs,
  author    = {Yuda Song, Zehao Sun, Xuanwu Yin},
  title     = {SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions},
  journal   = {arxiv},
  year      = {2024},
}