Sdxl Lora Planet Textures

🚀 SDXL LoRA for Planet Textures

This project provides LoRA (Low-Rank Adaptation) weights for fine - tuning the StableDiffusionXL model to generate planet - related textures. It offers a way to create diverse and realistic planet images based on text prompts.

🚀 Quick Start

Prerequisites

• Ensure you have torch, diffusers, and other necessary libraries installed.

Installation

The installation mainly involves setting up the environment and downloading the relevant models. You can use the following code to set up the pipeline:

import torch
from diffusers import DiffusionPipeline, AutoencoderKL

vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", vae=vae, torch_dtype=torch.float16, variant="fp16", use_safetensors=True)
pipe.load_lora_weights("sshh12/sdxl-lora-planet-textures")
pipe.to("cuda")

Usage

After setting up the pipeline, you can generate images using text prompts. Here is an example:

prompt = "A dwarf planet exhibiting a striking purple color, with a surface peppered with craters and towering ice formations"
negative_prompt = 'blurry, fuzzy, low resolution, cartoon, painting'

image = pipe(prompt=prompt, negative_prompt=negative_prompt, width=1024, height=512).images[0]
image

✨ Features

• Fine - Tuned on Planet Textures: The LoRA weights are fine - tuned on the sshh12/planet - textures dataset, enabling the generation of high - quality planet - related images.
• Special VAE: A special VAE (madebyollin/sdxl - vae - fp16 - fix) is used for training to improve image quality.

📦 Installation

The installation steps are mainly included in the quick start section. You need to install the necessary Python libraries and download the pre - trained models.

💻 Usage Examples

Basic Usage

import torch
from diffusers import DiffusionPipeline, AutoencoderKL

vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", vae=vae, torch_dtype=torch.float16, variant="fp16", use_safetensors=True)
pipe.load_lora_weights("sshh12/sdxl-lora-planet-textures")
pipe.to("cuda")

prompt = "A dwarf planet exhibiting a striking purple color, with a surface peppered with craters and towering ice formations"
negative_prompt = 'blurry, fuzzy, low resolution, cartoon, painting'

image = pipe(prompt=prompt, negative_prompt=negative_prompt, width=1024, height=512).images[0]
image

Advanced Usage

You can adjust the parameters such as width, height, prompt, and negative_prompt according to your needs to generate different planet images.

📚 Documentation

Model Information

Generated Images Examples

Prompt	Image
A turquoise - hued gas giant, streaked with swirling white wisps of high - altitude clouds, wrapped in a thin, multicolored ring system	
A barren desert planet, coated in fine rusty - red sand, pockmarked with deep, dark craters, and cloaked in a thin, hazy atmosphere	
A small, icy moon, encased in a shell of pure white ice and dust, covered with intricate patterns of frosty fissures
A metallic asteroid, featuring a rugged, heavily cratered surface with a shiny, silver - grey coloration	
A vibrant blue planet, teeming with lush, tropical forests and deep, azure oceans, surrounded by a thick, oxygen - rich atmosphere	
A dwarf planet exhibiting a striking purple color, with a surface peppered with craters and towering ice formations
A dusty, barren moon, characterized by a dull, yellowish - brown surface, marked by long, winding canyons and cliffs	

🔧 Technical Details

Training

The LoRA weights are trained using the following command:

MODEL_NAME="stabilityai/stable-diffusion-xl-base-1.0"
DATASET_NAME="sshh12/planet-textures"

!accelerate launch v2/train_text_to_image_lora_sdxl.py \
  --pretrained_model_name_or_path="$MODEL_NAME" \
  --pretrained_vae_model_name_or_path="madebyollin/sdxl-vae-fp16-fix" \
  --dataset_name="$DATASET_NAME" \
  --caption_column="text" \
  --width=1024 \
  --height=512 \
  --random_hflip \
  --random_vflip \
  --mixed_precision="fp16" \
  --use_8bit_adam \
  --train_batch_size=1 \
  --gradient_accumulation_steps=2 \
  --num_train_epochs=500 \
  --checkpointing_steps=100 \
  --learning_rate=1e-05 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --seed=0 \
  --validation_epochs=5 \
  --validation_prompt_file="v2/validation_prompts.txt" \
  --enable_xformers_memory_efficient_attention \
  --report_to="wandb"

GitHub Repository

You can find more details about the training process on the GitHub repository.

📄 License

This project is licensed under the CC - BY - NC - 4.0 license.