Cogvideox Disney Adamw 4000 0.0003 Constant

🚀 CogVideoX LoRA微调

这是一个针对文本到视频生成的项目，基于CogVideoX模型进行LoRA微调，能够利用特定数据集训练出具有特定风格的视频生成模型。

🚀 快速开始

本项目是 THUDM/CogVideoX - 5b 模型的LoRA微调版本。模型使用 CogVideoX Factory 进行训练，该仓库包含了使用 TorchAO 和 DeepSpeed 对CogVideoX系列模型进行内存优化的训练脚本。这些脚本改编自 CogVideoX Diffusers trainer。

✨ 主要特性

• 基于CogVideoX模型进行LoRA微调，可高效利用计算资源。
• 使用内存优化的训练脚本，适用于大规模数据集训练。
• 支持在Diffusers库中加载和使用LoRA权重。

📦 安装指南

使用此模型需要安装 🧨 Diffusers库。

💻 使用示例

基础用法

import torch
from diffusers import CogVideoXPipeline
from diffusers.utils import export_to_video

pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16).to("cuda")
pipe.load_lora_weights("a-r-r-o-w/cogvideox-disney-adamw-4000-0.0003-constant", weight_name="pytorch_lora_weights.safetensors", adapter_name="cogvideox-lora")

# The LoRA adapter weights are determined by what was used for training.
# In this case, we assume `--lora_alpha` is 32 and `--rank` is 64.
# It can be made lower or higher from what was used in training to decrease or amplify the effect
# of the LoRA upto a tolerance, beyond which one might notice no effect at all or overflows.
pipe.set_adapters(["cogvideox-lora"], [32 / 64])

video = pipe("BW_STYLE A black and white animated scene unfolds with an anthropomorphic goat surrounded by musical notes and symbols, suggesting a playful environment. Mickey Mouse appears, leaning forward in curiosity as the goat remains still. The goat then engages with Mickey, who bends down to converse or react. The dynamics shift as Mickey grabs the goat, potentially in surprise or playfulness, amidst a minimalistic background. The scene captures the evolving relationship between the two characters in a whimsical, animated setting, emphasizing their interactions and emotions", guidance_scale=6, use_dynamic_cfg=True).frames[0]
export_to_video(video, "output.mp4", fps=8)

更多详细信息，包括LoRA的加权、合并和融合，请查看 Diffusers中加载LoRA的文档。

📚 详细文档

下载模型

在 Files & Versions 标签中下载LoRA权重。

📄 许可证

请遵守 此处 和 此处 描述的许可条款。

信息表格