PixArt-Sigma-Nitro Open-Source Image Generation Model - Efficient Single-Step Inference, Effortlessly Convert Text to Images

Pixart Sigma Nitro

Developed by amd

AMD Nitro Diffusion is a series of efficient text-to-image models, distilled from mainstream diffusion models on AMD Instinct™ GPUs. PixArt-Sigma Nitro is a high-resolution single-step inference model based on Transformer architecture.

Image Generation Open Source License:Apache-2.0 #Single-step Diffusion Distillation #High-resolution Image Generation #AMD GPU Optimization

Downloads 21

Release Time : 11/5/2024

Model Overview

PixArt-Sigma Nitro is an efficient text-to-image model optimized through distillation from the PixArt-Sigma model, supporting single-step inference for high-quality image generation.

Model Features

Single-step Inference

The model is optimized through distillation to support single-step inference for image generation, significantly improving inference speed.

Efficient Performance

Compared to the original model, FLOPs are reduced by 90.9% with only minor sacrifices in CLIP and FID metrics.

Compatibility

Shares the same architecture as PixArt-Sigma and is compatible with diffusers workflows for easy integration and usage.

Model Capabilities

Text-to-Image Generation

High-resolution Image Generation

Single-step Inference

Use Cases

Creative Design

Rapid Concept Art Generation

Designers can quickly generate high-resolution concept art by inputting text prompts, improving workflow efficiency.

Single-step inference generates images, significantly reducing wait times.

Content Creation

Social Media Content Generation

Content creators can rapidly generate high-quality images tailored to themes for social media posts.

Maintains image quality while drastically improving generation speed.

🚀 AMD Nitro Diffusion

AMD Nitro Diffusion is a series of efficient text - to - image generation models. These models are distilled from popular diffusion models on AMD Instinct™ GPUs, offering high - performance image generation capabilities.

🚀 Quick Start

from diffusers import PixArtSigmaPipeline
import torch
from safetensors.torch import load_file

pipe = PixArtSigmaPipeline.from_pretrained("PixArt-alpha/PixArt-Sigma-XL-2-1024-MS")

ckpt_path = '<path to distilled checkpoint>'
transformer_state_dict = load_file(ckpt_path)
pipe.transformer.load_state_dict(transformer_state_dict)
pipe = pipe.to("cuda")

image = pipe(prompt='a photo of a cat',
             num_inference_steps=1,
             guidance_scale=0,
             timesteps=[400]).images[0]

For more details on training and evaluation please visit the GitHub repo.

✨ Features

AMD Nitro Diffusion consists of the following models:

Stable Diffusion 2.1 Nitro: a UNet - based one - step model distilled from Stable Diffusion 2.1.
PixArt - Sigma Nitro: a high - resolution transformer - based one - step model distilled from PixArt - Sigma.

⚡️ Open - source code! The models are based on our re - implementation of Latent Adversarial Diffusion Distillation, the method used to build the popular Stable Diffusion 3 Turbo model. Since the original authors didn't provide training code, we release our re - implementation to help advance further research in the field.

📚 Documentation

Details

Property	Details
Model Type	PixArt - Sigma Nitro has the same architecture as PixArt - Sigma and is compatible with the diffusers pipeline.
Inference Steps	This model is distilled to perform inference in just a single step. However, the training code also supports distilling a model for 2, 4 or 8 steps.
Hardware	We use a single node consisting of 4 AMD Instinct™ MI250 GPUs for distilling PixArt - Sigma Nitro.
Training Data	We use 1M prompts from DiffusionDB and generate the corresponding images from the base PixArt - Sigma model.
Training Cost	The distillation process achieves reasonable results in less than 2 days on a single node.

Results

Compared to PixArt - Sigma, our model achieves a 90.9% reduction in FLOPs at the cost of just 3.7% lower CLIP score and 10.5% higher FID.

Model	FID ↓	CLIP ↑	FLOPs	Latency on AMD Instinct MI250 (sec)
PixArt - Sigma, 20 steps	34.14	0.3289	187.96	7.46
PixArt - Sigma Nitro, 1 step	37.75	0.3167	17.04	0.53

📄 License

Copyright (c) 2018 - 2024 Advanced Micro Devices, Inc. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE - 2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

![image/jpeg](https://cdn - uploads.huggingface.co/production/uploads/6355aded9c72a7e742f341a4/AsUvS7acUDLZhKOMRSH37.jpeg)

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご