Wan2.1 T2V 1.3B 3dgs V0

🚀 文本到視頻LoRA微調模型

本項目是基於特定數據集對文本到視頻模型進行LoRA微調的成果，可用於生成具有特定風格的視頻內容。通過微調，模型能夠在特定場景下生成更符合需求的視頻。

🚀 快速開始

這是在 finetrainers/3dgs-dissolve 數據集上對 Wan-AI/Wan2.1-T2V-1.3B-Diffusers 模型進行的LoRA微調。

代碼倉庫：https://github.com/a-r-r-o-w/finetrainers

推理代碼

import torch
from diffusers import WanPipeline
from diffusers.utils import export_to_video

pipe = WanPipeline.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", torch_dtype=torch.bfloat16
).to("cuda")
pipe.load_lora_weights("finetrainers/Wan2.1-T2V-1.3B-3dgs-v0", adapter_name="wan-lora")
pipe.set_adapters(["wan-lora"], [0.5])

video = pipe("<my-awesome-prompt>").frames[0]
export_to_video(video, "output.mp4", fps=24)

訓練日誌可在WandB上查看：https://wandb.ai/aryanvs/finetrainers-wan

📦 模型信息

📚 示例展示

示例1

• 輸入文本：3DGS_DISSOLVE A lone sailor grips the helm as towering waves crash against the boat. Lightning splits the sky, illuminating the chaotic sea. The storm intensifies, but then, as if merging with the elements, the sailor's figure slowly transforms into a cascade of red sparks, scattering into the wind.
• 輸出視頻：3DGS_DISSOLVE_A_lone_sail-81.mp4

示例2

• 輸入文本：3DGS_DISSOLVE A snowboarder carves through fresh powder, the mountain landscape unfolding in crisp whites and deep blues. They launch into the air, spinning weightlessly before their form dissipates into a dazzling explosion of glowing embers that swirl into the icy breeze.
• 輸出視頻：3DGS_DISSOLVE_A_snowboard-81.mp4

示例3

• 輸入文本：3DGS_DISSOLVE A stark white bowl floats in darkness, its form slowly twisting. It begins to crack apart, glowing embers spilling outward, until it evaporates into a breathtaking eruption of red sparks. The scene transitions—now, a bouquet of flowers stands in its place, untouched, frozen in time.
• 輸出視頻：3DGS_DISSOLVE_A_stark_whi-81.mp4

示例4

• 輸入文本：3DGS_DISSOLVE A surfer carves through the curling wave, sunlight glinting off the water's surface. The tunnel of shimmering liquid twists around them, and just as they reach the crest, their form flickers—evaporating into a dazzling explosion of fiery sparks that vanish into the ocean spray.
• 輸出視頻：3DGS_DISSOLVE_A_surfer_ca-81.mp4

⚠️ 重要提示

⚠️ 重要提示

這是一個實驗性的檢查點，其泛化能力較差是已知的問題。