More examples are here: https://huggingface.co/datasets/piebro/factorio-blueprint-visualizations-sdxl-lora-examples.

Model description

Create abstract art in the style of artfully visualized Factorio blueprints. This Lora is trained using this dataset: https://huggingface.co/datasets/piebro/factorio-blueprint-visualizations.

Trigger words

You should use factorio to trigger the image generation.

Download model

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.

Replication

This is the command I used using for training the lora, using this file.

accelerate launch train_dreambooth_lora_sdxl.py --pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0 --pretrained_vae_model_name_or_path=madebyollin/sdxl-vae-fp16-fix --instance_data_dir=factorio_lora_images --instance_prompt='factorio' --output_dir=factorio_lora --seed=0 --resolution=1024 --train_batch_size=2 --max_train_steps=500 --checkpointing_steps=100 --gradient_accumulation_steps=2 --gradient_checkpointing --learning_rate=1e-4 --lr_scheduler=constant --lr_warmup_steps=0 --use_8bit_adam --mixed_precision=fp16

Usage

import torch
from diffusers import DiffusionPipeline, AutoencoderKL
device = "cuda" if torch.cuda.is_available() else "cpu"
vae = AutoencoderKL.from_pretrained('madebyollin/sdxl-vae-fp16-fix', torch_dtype=torch.float16)
pipe = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    vae=vae, torch_dtype=torch.float16, variant="fp16",
    use_safetensors=True
)
pipe.to(device)
# This is where you load your trained weights
specific_safetensors = "pytorch_lora_weights.safetensors"
lora_scale = 0.9
pipe.load_lora_weights(
    'piebro/factorio-blueprint-visualizations-sdxl-lora', 
    weight_name = specific_safetensors,
)
prompt = "pastel drawing of factorio"
image = pipe(
    prompt=prompt, 
    num_inference_steps=50,
    cross_attention_kwargs={"scale": lora_scale}
).images[0]

Upscaling

The modelx2 upscaler from https://huggingface.co/spaces/bookbot/Image-Upscaling-Playground worked quite well for upscaling the images. I used that to upscale some images for printing.

Example Prompts

lines

Patent drawing of factorio
graphite sketch of factorio
factorio, detailed architectural blueprint style
factorio, intricate pen and ink line art
factorio in a detailed scientific illustration style
factorio, concept sketch
factorio, scientific illustration
Technical schematic of factorio in the style of a detailed patent drawing
factorio depicted in a detailed architectural blueprint style, with precise measurements and labels
factorio in a minimalist, line art drawing

special

charcoal drawing of factorio
factorio in watercolor style
factorio, ball-point pen art
factorio visualized in a moody chiaroscuro ink drawing
factorio in a neon-lit digital cyberpunk art style
factorio, Art Deco-style geometric pattern design
factorio in a futuristic, sci-fi digital art concept
factorio, pixel art
factorio, pixel art, 8-bit colors
factorio, retro 1980s style digital pixel art
factorio by Rothko
factorio by Klimt
factorio by Ai Weiwei

normal

factorio
a texture of factorio
Pastel drawing of factorio
Pastel drawing of factorio, symetry
factorio, symetry
factorio, fractals
factorio, pattern
factorio, geometric pattern design
factorio, design
factorio, acrylic paint
factorio, Comic Book
factorio rendered in a photorealistic digital art style
factorio, vintage
factorio by Goya