Wan2.1 T2V 1.3B 3dgs V0

🚀 Wan2.1-T2V-1.3B LoRA Fine-tune

This project is a LoRA fine-tune of the Wan2.1-T2V-1.3B model, enabling more diverse text - to - video generation.

🚀 Quick Start

This is a LoRA fine-tune of the Wan-AI/Wan2.1-T2V-1.3B-Diffusers model on the finetrainers/3dgs-dissolve dataset.

✨ Features

• Text-to-Video: Capable of generating videos based on text descriptions.
• Diffusers Training: Utilizes the diffusers library for training.
• LoRA Fine-tune: Fine-tunes the base model using LoRA technology.

📦 Installation

The installation details are not provided in the original document. You may refer to the official documentation of the diffusers library and the base model for installation guidance.

💻 Usage Examples

Basic Usage

import torch
from diffusers import WanPipeline
from diffusers.utils import export_to_video

pipe = WanPipeline.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", torch_dtype=torch.bfloat16
).to("cuda")
pipe.load_lora_weights("finetrainers/Wan2.1-T2V-1.3B-3dgs-v0", adapter_name="wan-lora")
pipe.set_adapters(["wan-lora"], [0.5])

video = pipe("<my-awesome-prompt>").frames[0]
export_to_video(video, "output.mp4", fps=24)

📚 Documentation

Model Information

Widget Examples

• Example 1
  • Text: 3DGS_DISSOLVE A lone sailor grips the helm as towering waves crash against the boat. Lightning splits the sky, illuminating the chaotic sea. The storm intensifies, but then, as if merging with the elements, the sailor's figure slowly transforms into a cascade of red sparks, scattering into the wind.
  • Output: 3DGS_DISSOLVE_A_lone_sail-81.mp4
• Example 2
  • Text: 3DGS_DISSOLVE A snowboarder carves through fresh powder, the mountain landscape unfolding in crisp whites and deep blues. They launch into the air, spinning weightlessly before their form dissipates into a dazzling explosion of glowing embers that swirl into the icy breeze.
  • Output: 3DGS_DISSOLVE_A_snowboard-81.mp4
• Example 3
  • Text: 3DGS_DISSOLVE A stark white bowl floats in darkness, its form slowly twisting. It begins to crack apart, glowing embers spilling outward, until it evaporates into a breathtaking eruption of red sparks. The scene transitions—now, a bouquet of flowers stands in its place, untouched, frozen in time.
  • Output: 3DGS_DISSOLVE_A_stark_whi-81.mp4
• Example 4
  • Text: 3DGS_DISSOLVE A surfer carves through the curling wave, sunlight glinting off the water's surface. The tunnel of shimmering liquid twists around them, and just as they reach the crest, their form flickers—evaporating into a dazzling explosion of fiery sparks that vanish into the ocean spray.
  • Output: 3DGS_DISSOLVE_A_surfer_ca-81.mp4

Tags

• text-to-video
• diffusers-training
• diffusers
• template:sd-lora
• wan

📄 License

This project is under the other license.

⚠️ Important Note

This is an experimental checkpoint and its poor generalization is well-known.

Training logs are available on WandB here.