DFoT Open-Source Video Diffusion Model - Freely Generate High-Quality Videos Based on Multiple Context Frames

Dfot

Developed by kiwhansong

A novel video diffusion model capable of generating high-quality videos from any number of context frames

Video Processing Open Source License:MIT #Video Diffusion Generation #Multi-frame Conditional Generation #Long Video Stability

Downloads 47.19k

Release Time : 2/7/2025

Model Overview

DFoT is an innovative video generation model that incorporates History Guidance (HG) methods, significantly improving video generation quality, temporal consistency, and motion dynamics. It supports generating short videos or extremely long videos from single or multiple frames.

Model Features

History Guidance (HG) Method

DFoT's unique guidance technique that significantly enhances video generation quality, temporal consistency, and motion dynamics

Flexible Context Input

Supports video generation from any number of context frames (including single images)

Long Video Generation Capability

Capable of generating videos longer than 10 seconds, even supporting infinite navigation video generation

Composite Video Generation

Supports combining different video clips to generate coherent long videos

Model Capabilities

Generate videos from single images

Generate videos from multiple frames

Generate long-term stable videos

Composite video generation

Video motion control

Use Cases

Video Creation

Short Video Generation

Generate 2-second short videos from 1-2 images

High-quality, temporally consistent short video clips

Long Video Generation

Generate videos longer than 10 seconds from a single image

Dynamically stable and coherent long videos

Infinite Navigation Video

Generate infinitely extending navigation videos from a single image

Continuously stable, boundary-free video streams

Content Editing

Video Composition

Combine different video clips to generate coherent new videos

Seamlessly integrated composite videos

🚀 Diffusion Forcing Transformer

This is the official model hub for the paper History-guided Video Diffusion. It introduces a novel video diffusion model and guidance methods to enhance video generation.

Authors

*Equal contribution ¹MIT ²CMU ³Harvard

Links

This is the official model hub for the paper History-guided Video Diffusion. We introduce the Diffusion Forcing Tranformer (DFoT), a novel video diffusion model that designed to generate videos conditioned on an arbitrary number of context frames. Additionally, we present History Guidance (HG), a family of guidance methods uniquely enabled by DFoT. These methods significantly enhance video generation quality, temporal consistency, and motion dynamics, while also unlocking new capabilities such as compositional video generation and the stable rollout of extremely long videos.

image/png

🚀 Quick Start

🤗 Try generating videos with DFoT!

We provide an interactive demo on HuggingFace Spaces, where you can generate videos with DFoT and History Guidance. On the RealEstate10K dataset, you can generate:

Any Number of Images → Short 2 - second Video
Single Image → Long 10 - second Video
Single Image → Endless Navigation Video (like the teaser above!)

Please check it out and have fun generating videos with DFoT!

📦 Installation

All pretrained models can be automatically loaded from our GitHub codebase. Please visit our repository for further instructions!

📚 Documentation

📌 Citation

If our work is useful for your research, please consider citing our paper:

@misc{song2025historyguidedvideodiffusion,
  title={History - Guided Video Diffusion}, 
  author={Kiwhan Song and Boyuan Chen and Max Simchowitz and Yilun Du and Russ Tedrake and Vincent Sitzmann},
  year={2025},
  eprint={2502.06764},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2502.06764}, 
}

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご