Wan2.1 T2V 1.3B 3dgs V0

🚀 文本到视频LoRA微调模型

本项目是基于特定数据集对文本到视频模型进行LoRA微调的成果，可用于生成具有特定风格的视频内容。通过微调，模型能够在特定场景下生成更符合需求的视频。

🚀 快速开始

这是在 finetrainers/3dgs-dissolve 数据集上对 Wan-AI/Wan2.1-T2V-1.3B-Diffusers 模型进行的LoRA微调。

代码仓库：https://github.com/a-r-r-o-w/finetrainers

推理代码

import torch
from diffusers import WanPipeline
from diffusers.utils import export_to_video

pipe = WanPipeline.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", torch_dtype=torch.bfloat16
).to("cuda")
pipe.load_lora_weights("finetrainers/Wan2.1-T2V-1.3B-3dgs-v0", adapter_name="wan-lora")
pipe.set_adapters(["wan-lora"], [0.5])

video = pipe("<my-awesome-prompt>").frames[0]
export_to_video(video, "output.mp4", fps=24)

训练日志可在WandB上查看：https://wandb.ai/aryanvs/finetrainers-wan

📦 模型信息

📚 示例展示

示例1

• 输入文本：3DGS_DISSOLVE A lone sailor grips the helm as towering waves crash against the boat. Lightning splits the sky, illuminating the chaotic sea. The storm intensifies, but then, as if merging with the elements, the sailor's figure slowly transforms into a cascade of red sparks, scattering into the wind.
• 输出视频：3DGS_DISSOLVE_A_lone_sail-81.mp4

示例2

• 输入文本：3DGS_DISSOLVE A snowboarder carves through fresh powder, the mountain landscape unfolding in crisp whites and deep blues. They launch into the air, spinning weightlessly before their form dissipates into a dazzling explosion of glowing embers that swirl into the icy breeze.
• 输出视频：3DGS_DISSOLVE_A_snowboard-81.mp4

示例3

• 输入文本：3DGS_DISSOLVE A stark white bowl floats in darkness, its form slowly twisting. It begins to crack apart, glowing embers spilling outward, until it evaporates into a breathtaking eruption of red sparks. The scene transitions—now, a bouquet of flowers stands in its place, untouched, frozen in time.
• 输出视频：3DGS_DISSOLVE_A_stark_whi-81.mp4

示例4

• 输入文本：3DGS_DISSOLVE A surfer carves through the curling wave, sunlight glinting off the water's surface. The tunnel of shimmering liquid twists around them, and just as they reach the crest, their form flickers—evaporating into a dazzling explosion of fiery sparks that vanish into the ocean spray.
• 输出视频：3DGS_DISSOLVE_A_surfer_ca-81.mp4

⚠️ 重要提示

⚠️ 重要提示

这是一个实验性的检查点，其泛化能力较差是已知的问题。