Onlyflow

🚀 OnlyFlow - Obvious Research

OnlyFlow is a model for video diffusion, leveraging optical flow for motion conditioning. It offers high - quality video generation and is trained on large - scale datasets.

🚀 Quick Start

You can quickly start using OnlyFlow through the following ways:

• Online Interface: Use the gradio interface on our HuggingFace Space.
• Local Testing: Use the inference script in our Github repository to test the released model.

⚠️ Important Note

The model is really sensitive to the input video framerate. You should try downsampling the video to 8fps before using it if you don't get the desired results.

✨ Features

• Optical Flow - Based Conditioning: The model is based on the research of OnlyFlow: Optical Flow based Motion Conditioning for Video Diffusion Models, using optical flow for motion conditioning in video diffusion.
• Multiple Output Options: It can generate different video outputs by adjusting the gamma parameter, as shown in the examples below.
• Multiple Model Weight Formats: We release the model weights in fp32, fp16 ckpt and safetensors format.

📦 Installation

The README does not provide specific installation steps, so this section is skipped.

💻 Usage Examples

Basic Usage

You can use the gradio interface on our HuggingFace Space for basic inference.

Advanced Usage

You can use the inference script in our Github repository to test the released model. This allows for more customized testing and research.

📚 Documentation

Model Information

Model Weight Release

We release the model weights of our best training in fp32, fp16 ckpt and safetensors format. The model is trained on the Webvid - 10M dataset on an 8 A100 node for 20h.

Output Examples

Here are examples of videos created by the model, with from left to right:

• input
• optical flow extracted from input
• output with onylflow influence (gamma parameter set to 0.)
• output with gamma = 0.5
• output with gamma = 0.75
• output with gamma = 1.

Project Links

• Project Website: website
• Code Repository: Github
• Model Weights: HuggingFace

🔧 Technical Details

The README does not provide detailed technical implementation details, so this section is skipped.

📄 License

The model is released under the apache - 2.0 license.

Next Steps

We are working on improving OnlyFlow and train it for other base model than AnimateDiff.

We appreciate any help, feel free to reach out! You can contact us:

• On Twitter: @obv_research
• By mail: research.obvious@gmail.com

About Obvious Research

Obvious Research is an Artificial Intelligence research laboratory dedicated to creating new AI artistic tools, initiated by the artists’ trio Obvious, in partnership with La Sorbonne Université.