Pony Diffusion

🚀 pony-diffusion - >nohooves

Pony Diffusion V4 is now live!

pony-diffusion is a latent text-to-image diffusion model. It has been fine - tuned on high - quality pony SFW - ish images. This model can generate pony - related images based on text prompts.

Special thanks go to Waifu - Diffusion for finetuning expertise and Novel AI for providing necessary compute.

Original PyTorch Model Download Link

Real - ESRGAN Model finetuned on pony faces

📚 Documentation

Model Description

The model for fine - tuning is an early finetuned checkpoint of waifu - diffusion on top of Stable Diffusion V1 - 4. Stable Diffusion V1 - 4 is a latent image diffusion model trained on LAION2B - en.

This checkpoint was fine - tuned with a learning rate of 5.0e - 6 for 4 epochs on about 80k pony text - image pairs. These pairs use tags from derpibooru, have scores greater than 500, and belong to the safe or suggestive categories.

📄 License

This model is open access. It uses the CreativeML OpenRAIL - M license to specify rights and usage. The CreativeML OpenRAIL License states:

1. You can't use the model to deliberately produce or share illegal or harmful outputs or content.
2. The authors claim no rights on the outputs you generate. You are free to use them but accountable for their use, which must comply with the license.
3. You may re - distribute the weights and use the model commercially or as a service. If so, include the same use restrictions as in the license and share a copy of the CreativeML OpenRAIL - M with all users. Please read the full license here

💻 Usage Examples

Basic Usage

import torch
from torch import autocast
from diffusers import StableDiffusionPipeline, DDIMScheduler
model_id = "AstraliteHeart/pony-diffusion"
device = "cuda"
pipe = StableDiffusionPipeline.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    revision="fp16",
    scheduler=DDIMScheduler(
        beta_start=0.00085,
        beta_end=0.012,
        beta_schedule="scaled_linear",
        clip_sample=False,
        set_alpha_to_one=False,
    ),
)
pipe = pipe.to(device)
prompt = "pinkie pie anthro portrait wedding dress veil intricate highly detailed digital painting artstation concept art smooth sharp focus illustration Unreal Engine 5 8K"
with autocast("cuda"):
    image = pipe(prompt, guidance_scale=7.5)["sample"][0]  
    
image.save("cute_poner.png")

🔧 Technical Details

Downstream Uses

This model can be used for entertainment and as a generative art assistant.

Team Members and Acknowledgements

This project wouldn't be possible without the work of CompVis Researchers.

• Thanks to Waifu - Diffusion for finetuning help and the starting checkpoint.
• Thanks to Novel AI for providing compute.

To contact us, join our Discord server.