Df Cpt Mo Di Bear Guitar

🚀 Tune-A-Video - 現代迪士尼風格

本項目是一個與Diffusers兼容的模型，可用於文本到視頻的生成，基於現代迪士尼風格，通過特定訓練提示進行訓練，能生成具有特色的視頻內容。

🚀 快速開始

本模型是一個與Diffusers兼容的檢查點。當與DiffusionPipeline一起使用時，會返回一個TuneAVideoPipeline實例。

df-cpt 用於表示它是Tune-A-Video-library/mo-di-bear-guitar的Diffusers兼容版本。

• 基礎模型：nitrosocke/mo-di-diffusion
• 訓練提示：一隻熊在彈吉他。

✨ 主要特性

• 與Diffusers庫兼容，方便集成到現有的擴散模型流程中。
• 基於現代迪士尼風格，能生成具有該風格特色的視頻內容。

💻 使用示例

基礎用法

使用現有的文本到圖像檢查點加載

import torch
from diffusers import TuneAVideoPipeline, DDIMScheduler, UNet3DConditionModel
from diffusers.utils import export_to_video
from PIL import Image

# Use any pretrained Text2Image checkpoint based on stable diffusion
pretrained_model_path = "nitrosocke/mo-di-diffusion"
unet = UNet3DConditionModel.from_pretrained(
    "Tune-A-Video-library/df-cpt-mo-di-bear-guitar", subfolder="unet", torch_dtype=torch.float16
).to("cuda")

pipe = TuneAVideoPipeline.from_pretrained(pretrained_model_path, unet=unet, torch_dtype=torch.float16).to("cuda")

prompt = "A princess playing a guitar, modern disney style"
generator = torch.Generator(device="cuda").manual_seed(42)

video_frames = pipe(prompt, video_length=3, generator=generator, num_inference_steps=50, output_type="np").frames

# Saving to gif.
pil_frames = [Image.fromarray(frame) for frame in video_frames]
duration = len(pil_frames) / 8
pil_frames[0].save(
    "animation.gif",
    save_all=True,
    append_images=pil_frames[1:],  # append rest of the images
    duration=duration * 1000,  # in milliseconds
    loop=0,
)

# Saving to video
video_path = export_to_video(video_frames)

高級用法

加載已保存的Tune-A-Video檢查點

import torch
from diffusers import DiffusionPipeline, DDIMScheduler
from diffusers.utils import export_to_video
from PIL import Image

pipe = DiffusionPipeline.from_pretrained(
    "Tune-A-Video-library/df-cpt-mo-di-bear-guitar", torch_dtype=torch.float16
).to("cuda")

prompt = "A princess playing a guitar, modern disney style"
generator = torch.Generator(device="cuda").manual_seed(42)

video_frames = pipe(prompt, video_length=3, generator=generator, num_inference_steps=50, output_type="np").frames

# Saving to gif.
pil_frames = [Image.fromarray(frame) for frame in video_frames]
duration = len(pil_frames) / 8
pil_frames[0].save(
    "animation.gif",
    save_all=True,
    append_images=pil_frames[1:],  # append rest of the images
    duration=duration * 1000,  # in milliseconds
    loop=0,
)

# Saving to video
video_path = export_to_video(video_frames)

📚 詳細文檔

示例

測試提示： "一位公主在彈吉他，現代迪士尼風格"

相關論文

• Tune-A-Video：One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
• Stable Diffusion：High-Resolution Image Synthesis with Latent Diffusion Models

📄 許可證

本模型使用的許可證為 creativeml-openrail-m。