Cartoonify

🚀 Cartoonify

This is a DreamBooth model based on runwayml/stable-diffusion-v1-5, with additional fine - tuning of the text encoder. It can generate images in a Disney - like style.

🚀 Quick Start

This DreamBooth model is derived from runwayml/stable-diffusion-v1-5 and fine - tuned on the text encoder. The weights are trained using DreamBooth with data from a well - known animation studio. Use the token disney style in your prompts to get the desired effect.

You can find some example images below:

✨ Features

• Based on Stable Diffusion v1 - 5, with enhanced text - to - image generation capabilities.
• Capable of generating images in a Disney - like style.

💻 Usage Examples

Basic Usage

import torch
from diffusers import StableDiffusionPipeline

# basic usage
repo_id = "lavaman131/cartoonify"
device = torch.device("cuda")
torch_dtype = torch.float16 if device.type in ["mps", "cuda"] else torch.float32
pipeline = StableDiffusionPipeline.from_pretrained(repo_id, torch_dtype=torch_dtype).to(device)
image = pipeline("PROMPT GOES HERE").images[0]
image.save("output.png")

Advanced Usage

The full source - code used for training and local gradio demo for image to disney character style transfer can be found here.

🔧 Technical Details

The model was fine - tuned for 3500 steps on around 200 images of modern Disney characters, backgrounds, and animals. The ratios for each were 70%, 20%, and 10% respectively on an RTX A5000 GPU (24GB VRAM).

The training code used can be found here. The regularization images used for training can be found [here](https://github.com/aitrepreneur/SD - Regularization - Images - Style - Dreambooth/tree/main/style_ddim).

📄 License

This model uses the creativeml-openrail-m license.

📚 Documentation

Intended uses & limitations

How to use

Refer to the code example in the "Usage Examples" section.

Limitations and bias

As with any diffusion model, you need to adjust the prompt and classifier - free guidance parameter until you get the desired results. Zoomed - out subjects may lose clarity in the face. For additional safety in image generation, we use the Stable Diffusion safety checker.