Show 1 Sr2

🚀 Show-1 SR2

Pixel-based VDMs can precisely generate motion that aligns with textual prompts. However, they usually incur high computational costs in terms of time and GPU memory, especially when generating high-resolution videos. In contrast, latent-based VDMs are more resource-efficient as they operate in a reduced-dimension latent space. Nevertheless, it is difficult for such a small latent space (e.g., 64×40 for 256×160 videos) to capture the rich visual semantic details described by the textual prompt.

To leverage the strengths and mitigate the weaknesses of pixel-based and latent-based VDMs, we present Show-1, an efficient text-to-video model that generates videos with both good video-text alignment and high visual quality.

📚 Documentation

Model Details

This is the super-resolution model of Show-1, which upscales videos from a 256x160 resolution to 576x320. The model is finetuned using diffusion timesteps 0 - 900 on the WebVid-10M dataset.

📦 Installation

Clone the GitHub repository and install the requirements:

git clone https://github.com/showlab/Show-1.git
pip install -r requirements.txt

💻 Usage Examples

Basic Usage

Run the following command to generate a video from a text prompt. By default, this will automatically download all the model weights from huggingface.

python run_inference.py

Advanced Usage

You can also download the weights manually and change the pretrained_model_path in run_inference.py to run the inference.

git lfs install

# base
git clone https://huggingface.co/showlab/show-1-base
# interp
git clone https://huggingface.co/showlab/show-1-interpolation
# sr1
git clone https://huggingface.co/showlab/show-1-sr1
# sr2
git clone https://huggingface.co/showlab/show-1-sr2

📄 License

This model is released under the Creative Commons Attribution Non Commercial 4.0 license.

📚 Citation

If you make use of our work, please cite our paper.

@misc{zhang2023show1,
    title={Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation}, 
    author={David Junhao Zhang and Jay Zhangjie Wu and Jia-Wei Liu and Rui Zhao and Lingmin Ran and Yuchao Gu and Difei Gao and Mike Zheng Shou},
    year={2023},
    eprint={2309.15818},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

📞 Model Card Contact

This model card is maintained by David Junhao Zhang and Jay Zhangjie Wu. For any questions, please feel free to contact us or open an issue in the repository.