Df Cpt Mo Di Bear Guitar

🚀 Tune-A-Video - Modern Disney

This is a diffusers compatible checkpoint for generating modern Disney-style videos from text prompts.

🚀 Quick Start

This project offers a diffusers - compatible checkpoint. When used with the DiffusionPipeline, it returns an instance of the TuneAVideoPipeline.

df - cpt is used to indicate that it's a diffusers compatible equivalent of Tune - A - Video - library/mo - di - bear - guitar.

✨ Features

• Base model: [nitrosocke/mo - di - diffusion](https://huggingface.co/nitrosocke/mo - di - diffusion)
• Training prompt: a bear is playing guitar.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

Loading with a pre - existing Text2Image checkpoint

import torch
from diffusers import TuneAVideoPipeline, DDIMScheduler, UNet3DConditionModel
from diffusers.utils import export_to_video
from PIL import Image

# Use any pretrained Text2Image checkpoint based on stable diffusion
pretrained_model_path = "nitrosocke/mo-di-diffusion"
unet = UNet3DConditionModel.from_pretrained(
    "Tune-A-Video-library/df-cpt-mo-di-bear-guitar", subfolder="unet", torch_dtype=torch.float16
).to("cuda")

pipe = TuneAVideoPipeline.from_pretrained(pretrained_model_path, unet=unet, torch_dtype=torch.float16).to("cuda")

prompt = "A princess playing a guitar, modern disney style"
generator = torch.Generator(device="cuda").manual_seed(42)

video_frames = pipe(prompt, video_length=3, generator=generator, num_inference_steps=50, output_type="np").frames

# Saving to gif.
pil_frames = [Image.fromarray(frame) for frame in video_frames]
duration = len(pil_frames) / 8
pil_frames[0].save(
    "animation.gif",
    save_all=True,
    append_images=pil_frames[1:],  # append rest of the images
    duration=duration * 1000,  # in milliseconds
    loop=0,
)

# Saving to video
video_path = export_to_video(video_frames)

Advanced Usage

Loading a saved Tune - A - Video checkpoint

import torch
from diffusers import DiffusionPipeline, DDIMScheduler
from diffusers.utils import export_to_video
from PIL import Image

pipe = DiffusionPipeline.from_pretrained(
    "Tune-A-Video-library/df-cpt-mo-di-bear-guitar", torch_dtype=torch.float16
).to("cuda")

prompt = "A princess playing a guitar, modern disney style"
generator = torch.Generator(device="cuda").manual_seed(42)

video_frames = pipe(prompt, video_length=3, generator=generator, num_inference_steps=50, output_type="np").frames

# Saving to gif.
pil_frames = [Image.fromarray(frame) for frame in video_frames]
duration = len(pil_frames) / 8
pil_frames[0].save(
    "animation.gif",
    save_all=True,
    append_images=pil_frames[1:],  # append rest of the images
    duration=duration * 1000,  # in milliseconds
    loop=0,
)

# Saving to video
video_path = export_to_video(video_frames)

📚 Documentation

Samples

Test prompt: "A princess playing a guitar, modern disney style"

Related Papers

• Tune - A - Video: One - Shot Tuning of Image Diffusion Models for Text - to - Video Generation
• Stable Diffusion: High - Resolution Image Synthesis with Latent Diffusion Models

📄 License

• License: creativeml - openrail - m