Wan Flat Color V2

🚀 Flat Color - Style

This project is designed for text - to - image and text - to - video generation, leveraging LoRA and Diffusers. It offers a unique flat - color, no - lineart style.

🚀 Quick Start

Using with Diffusers

First, install the required library:

pip install git+https://github.com/huggingface/diffusers.git

Then, you can use the following Python code to generate videos:

import torch
from diffusers.utils import export_to_video
from diffusers import AutoencoderKLWan, WanPipeline
from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler

# Available models: Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.1-T2V-1.3B-Diffusers
model_id = "Wan-AI/Wan2.1-T2V-14B-Diffusers"
vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16)
flow_shift = 5.0  # 5.0 for 720P, 3.0 for 480P
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config, flow_shift=flow_shift)
pipe.to("cuda")

pipe.load_lora_weights("motimalu/wan-flat-color-v2")

pipe.enable_model_cpu_offload() #for low-vram environments

prompt = "A cat wandering around new york city"
negative_prompt = "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"

output = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=480,
    width=720,
    num_frames=81,
    guidance_scale=5.0,
).frames[0]
export_to_video(output, "output.mp4", fps=16)

✨ Features

Model description

Flat Color - Style is trained on images without visible lineart, flat colors, and little to no indication of depth. Text to Video previews are generated with ComfyUI_examples/wan/#text-to-video. The LoRA is loaded with the LoraLoaderModelOnly node. It is reprinted from CivitAI by request: https://civitai.com/models/1132089?modelVersionId=1474944

Trigger words

You should use flat color and no lineart to trigger the image generation.

📦 Installation

The installation step is included in the "Using with Diffusers" section above. You need to run the following command to install the necessary library:

pip install git+https://github.com/huggingface/diffusers.git

📚 Documentation

Gallery