RCNA MINI

🚀 RCNA MINI

RCNA MINI 是一款紧凑的 LoRA（低秩自适应）模型，专为生成高质量的 4 步文本到视频输出而设计。它可以创建时长从 4 秒到 16 秒的视频片段，非常适合生成具有丰富细节和流畅过渡的短动画。

🚀 快速开始

RCNA MINI 是基于 LoRA 架构的模型，能够根据文本描述快速生成视频。以下是使用它的示例代码：

import torch
from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter, DiffusionPipeline
from diffusers.utils import export_to_gif

# Load AnimateLCM for video generation
adapter = MotionAdapter.from_pretrained("Binarybardakshat/RCNA_MINI")
pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
pipe.load_lora_weights("Binarybardakshat/RCNA_MINI", weight_name="RCNA_LORA_MINI_1.safetensors", adapter_name="lcm-lora")
pipe.set_adapters(["lcm-lora"], [0.8])
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

# Generate video using RCNA MINI
output = pipe(
    prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=16,
    guidance_scale=2.0,
    num_inference_steps=6,
    generator=torch.Generator("cpu").manual_seed(0),
)
frames = output.frames[0]
export_to_gif(frames, "animatelcm.gif")
print("Video and image generation complete!")

✨ 主要特性

• 4 步文本到视频：仅需 4 步即可根据文本提示生成视频。
• 视频长度：可生成 4 秒至 16 秒长的视频。
• 高质量：支持高分辨率和详细的输出（最高可达 8K）。
• 快速采样：利用解耦一致性学习，该模型在保证质量的同时优化了速度。

💻 使用示例

基础用法

# 上述快速开始中的代码即为基础用法示例
import torch
from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter, DiffusionPipeline
from diffusers.utils import export_to_gif

# Load AnimateLCM for video generation
adapter = MotionAdapter.from_pretrained("Binarybardakshat/RCNA_MINI")
pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
pipe.load_lora_weights("Binarybardakshat/RCNA_MINI", weight_name="RCNA_LORA_MINI_1.safetensors", adapter_name="lcm-lora")
pipe.set_adapters(["lcm-lora"], [0.8])
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

# Generate video using RCNA MINI
output = pipe(
    prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=16,
    guidance_scale=2.0,
    num_inference_steps=6,
    generator=torch.Generator("cpu").manual_seed(0),
)
frames = output.frames[0]
export_to_gif(frames, "animatelcm.gif")
print("Video and image generation complete!")

高级用法

# 可以根据不同的需求调整参数，例如修改提示词、帧数、引导比例等
import torch
from diffusers import AnimateDiffPipeline, LCMScheduler, MotionAdapter, DiffusionPipeline
from diffusers.utils import export_to_gif

# Load AnimateLCM for video generation
adapter = MotionAdapter.from_pretrained("Binarybardakshat/RCNA_MINI")
pipe = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear")
pipe.load_lora_weights("Binarybardakshat/RCNA_MINI", weight_name="RCNA_LORA_MINI_1.safetensors", adapter_name="lcm-lora")
pipe.set_adapters(["lcm-lora"], [0.8])
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

# 调整提示词和帧数
output = pipe(
    prompt="A beautiful forest with colorful flowers, 8k, high resolution",
    negative_prompt="bad quality, worse quality, low resolution",
    num_frames=20,
    guidance_scale=2.5,
    num_inference_steps=8,
    generator=torch.Generator("cpu").manual_seed(1),
)
frames = output.frames[0]
export_to_gif(frames, "advanced_animatelcm.gif")
print("Advanced video and image generation complete!")

📚 详细文档

示例输出

• 提示词："Astronaut in a jungle, cold color palette, muted colors, detailed, 8K"
  • 生成一个具有丰富细节和流畅运动的高质量视频。

工作原理

RCNA MINI 基于 LoRA 架构，该架构使用低秩自适应对扩散模型进行微调。与全模型重新训练相比，这使得生成速度更快，计算开销更小。

应用场景

• 社交媒体内容的短格式动画
• 创意项目的视频生成
• 基于文本描述的艺术视频生成

模型详情

📄 许可证

本模型遵循 MIT 许可证。