Stable Diffusion 3.5 Large DF11

🚀 DFloat11 Compressed Model: stabilityai/stable-diffusion-3.5-large

This is a losslessly compressed version of the stabilityai/stable-diffusion-3.5-large model using the custom DFloat11 format, offering significant size reduction without sacrificing output quality.

✨ Features

• Base Model: stabilityai/stable-diffusion-3.5-large
• Base Model Relation: Quantized
• Pipeline Tag: Text-to-image
• Tags: dfloat11, df11, lossless compression, 70% size, 100% accuracy

Key Benefits

• Bit-for-bit identical outputs: The outputs are exactly the same as the original BFloat16 model.
• ~30% reduction in model size: The model size is reduced from 16GB to 11.3GB.
• Lower memory requirements: It can now run on 16GB GPUs.
• Minimal performance overhead: It is barely slower than the full model.

DFloat11 compresses the model weights while maintaining full numerical precision. This enables you to run stabilityai/stable-diffusion-3.5-large on more accessible hardware without any compromise in output quality.

How It Works

DFloat11 compresses model weights using Huffman coding of BFloat16 exponent bits, combined with hardware-aware algorithmic designs for efficient on-the-fly decompression directly on the GPU. During inference, the weights stay compressed in GPU memory and are decompressed right before matrix multiplications, then immediately discarded after use to minimize the memory footprint.

Advantages:

• Fully GPU-based: There is no CPU decompression or host-device data transfer.
• Faster than CPU-offloading approaches: DFloat11 allows for practical deployment in memory-constrained environments.
• Fully lossless compression: The model's outputs are bit-for-bit identical to those of the original model.

📦 Installation

1. Install or upgrade the DFloat11 pip package (it installs the CUDA kernel automatically; a CUDA-compatible GPU and PyTorch are required):

pip install -U dfloat11[cuda12]
# or if you have CUDA version 11:
# pip install -U dfloat11[cuda11]

2. Install or upgrade the diffusers package:

pip install -U diffusers

💻 Usage Examples

Basic Usage

import torch
from diffusers import StableDiffusion3Pipeline
from dfloat11 import DFloat11Model

pipe = StableDiffusion3Pipeline.from_pretrained("stabilityai/stable-diffusion-3.5-large", torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

DFloat11Model.from_pretrained('DFloat11/stable-diffusion-3.5-large-DF11', device='cpu', bfloat16_model=pipe.transformer)

image = pipe(
    "A capybara holding a sign that reads Hello World",
    num_inference_steps=28,
    guidance_scale=3.5,
).images[0]
image.save("capybara.png")

📚 Documentation

• Paper: 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
• GitHub: https://github.com/LeanModels/DFloat11
• HuggingFace: https://huggingface.co/DFloat11