Cogvideox Disney Adamw 4000 0.0003 Constant

🚀 CogVideoX LoRA微調

這是一個針對文本到視頻生成的項目，基於CogVideoX模型進行LoRA微調，能夠利用特定數據集訓練出具有特定風格的視頻生成模型。

🚀 快速開始

本項目是 THUDM/CogVideoX - 5b 模型的LoRA微調版本。模型使用 CogVideoX Factory 進行訓練，該倉庫包含了使用 TorchAO 和 DeepSpeed 對CogVideoX系列模型進行內存優化的訓練腳本。這些腳本改編自 CogVideoX Diffusers trainer。

✨ 主要特性

• 基於CogVideoX模型進行LoRA微調，可高效利用計算資源。
• 使用內存優化的訓練腳本，適用於大規模數據集訓練。
• 支持在Diffusers庫中加載和使用LoRA權重。

📦 安裝指南

使用此模型需要安裝 🧨 Diffusers庫。

💻 使用示例

基礎用法

import torch
from diffusers import CogVideoXPipeline
from diffusers.utils import export_to_video

pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16).to("cuda")
pipe.load_lora_weights("a-r-r-o-w/cogvideox-disney-adamw-4000-0.0003-constant", weight_name="pytorch_lora_weights.safetensors", adapter_name="cogvideox-lora")

# The LoRA adapter weights are determined by what was used for training.
# In this case, we assume `--lora_alpha` is 32 and `--rank` is 64.
# It can be made lower or higher from what was used in training to decrease or amplify the effect
# of the LoRA upto a tolerance, beyond which one might notice no effect at all or overflows.
pipe.set_adapters(["cogvideox-lora"], [32 / 64])

video = pipe("BW_STYLE A black and white animated scene unfolds with an anthropomorphic goat surrounded by musical notes and symbols, suggesting a playful environment. Mickey Mouse appears, leaning forward in curiosity as the goat remains still. The goat then engages with Mickey, who bends down to converse or react. The dynamics shift as Mickey grabs the goat, potentially in surprise or playfulness, amidst a minimalistic background. The scene captures the evolving relationship between the two characters in a whimsical, animated setting, emphasizing their interactions and emotions", guidance_scale=6, use_dynamic_cfg=True).frames[0]
export_to_video(video, "output.mp4", fps=8)

更多詳細信息，包括LoRA的加權、合併和融合，請查看 Diffusers中加載LoRA的文檔。

📚 詳細文檔

下載模型

在 Files & Versions 標籤中下載LoRA權重。

📄 許可證

請遵守 此處 和 此處 描述的許可條款。

信息表格