RCNA MINI

🚀 RCNA MINI

RCNA MINI 是一款緊湊的 LoRA（低秩自適應）模型，專為生成高質量的 4 步文本到視頻輸出而設計。它可以創建時長從 4 秒到 16 秒的視頻片段，非常適合生成具有豐富細節和流暢過渡的短動畫。

🚀 快速開始

RCNA MINI 是基於 LoRA 架構的模型，能夠根據文本描述快速生成視頻。以下是使用它的示例代碼：

import torch
from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter, DiffusionPipeline
from diffusers.utils import export_to_gif

# Load AnimateLCM for video generation
adapter = MotionAdapter.from_pretrained("Binarybardakshat/RCNA_MINI")
pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
pipe.load_lora_weights("Binarybardakshat/RCNA_MINI", weight_name="RCNA_LORA_MINI_1.safetensors", adapter_name="lcm-lora")
pipe.set_adapters(["lcm-lora"], [0.8])
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

# Generate video using RCNA MINI
output = pipe(
    prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=16,
    guidance_scale=2.0,
    num_inference_steps=6,
    generator=torch.Generator("cpu").manual_seed(0),
)
frames = output.frames[0]
export_to_gif(frames, "animatelcm.gif")
print("Video and image generation complete!")

✨ 主要特性

• 4 步文本到視頻：僅需 4 步即可根據文本提示生成視頻。
• 視頻長度：可生成 4 秒至 16 秒長的視頻。
• 高質量：支持高分辨率和詳細的輸出（最高可達 8K）。
• 快速採樣：利用解耦一致性學習，該模型在保證質量的同時優化了速度。

💻 使用示例

基礎用法

# 上述快速開始中的代碼即為基礎用法示例
import torch
from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter, DiffusionPipeline
from diffusers.utils import export_to_gif

# Load AnimateLCM for video generation
adapter = MotionAdapter.from_pretrained("Binarybardakshat/RCNA_MINI")
pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
pipe.load_lora_weights("Binarybardakshat/RCNA_MINI", weight_name="RCNA_LORA_MINI_1.safetensors", adapter_name="lcm-lora")
pipe.set_adapters(["lcm-lora"], [0.8])
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

# Generate video using RCNA MINI
output = pipe(
    prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=16,
    guidance_scale=2.0,
    num_inference_steps=6,
    generator=torch.Generator("cpu").manual_seed(0),
)
frames = output.frames[0]
export_to_gif(frames, "animatelcm.gif")
print("Video and image generation complete!")

高級用法

# 可以根據不同的需求調整參數，例如修改提示詞、幀數、引導比例等
import torch
from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter, DiffusionPipeline
from diffusers.utils import export_to_gif

# Load AnimateLCM for video generation
adapter = MotionAdapter.from_pretrained("Binarybardakshat/RCNA_MINI")
pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
pipe.load_lora_weights("Binarybardakshat/RCNA_MINI", weight_name="RCNA_LORA_MINI_1.safetensors", adapter_name="lcm-lora")
pipe.set_adapters(["lcm-lora"], [0.8])
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

# 調整提示詞和幀數
output = pipe(
    prompt="A beautiful forest with colorful flowers, 8k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=20,
    guidance_scale=2.5,
    num_inference_steps=8,
    generator=torch.Generator("cpu").manual_seed(1),
)
frames = output.frames[0]
export_to_gif(frames, "advanced_animatelcm.gif")
print("Advanced video and image generation complete!")

📚 詳細文檔

示例輸出

• 提示詞："Astronaut in a jungle, cold color palette, muted colors, detailed, 8K"
  • 生成一個具有豐富細節和流暢運動的高質量視頻。

工作原理

RCNA MINI 基於 LoRA 架構，該架構使用低秩自適應對擴散模型進行微調。與全模型重新訓練相比，這使得生成速度更快，計算開銷更小。

應用場景

• 社交媒體內容的短格式動畫
• 創意項目的視頻生成
• 基於文本描述的藝術視頻生成

模型詳情

📄 許可證

本模型遵循 MIT 許可證。