đ Text2Video-Zero Model Card - ControlNet Canny Avatar Style
Text2Video-Zero is a zero-shot text to video generator. It can perform various tasks such as zero-shot text-to-video generation, Video Instruct Pix2Pix (instruction-guided video editing), text and pose conditional video generation, text and canny-edge conditional video generation, and text, canny-edge and dreambooth conditional video generation. For more details, check our paper and our demo: 
. Our code is compatible with any StableDiffusion base model. This model offers DreamBooth weights for the Avatar style for use with edge guidance (using ControlNet) in text2video zero.
đ Quick Start
This section provides an overview of the Text2Video - Zero model and its capabilities. The model can generate videos from text prompts and perform multiple video - related tasks.
⨠Features
- Zero - shot Text - to - Video Generation: Generate videos directly from text prompts without extensive fine - tuning.
- Video Editing: Perform instruction - guided video editing with
Video Instruct Pix2Pix.
- Conditional Video Generation: Generate videos based on text, pose, canny - edge, and dreambooth conditions.
đ Documentation
Weights for Text2Video - Zero
We converted the original weights into diffusers and made them usable for ControlNet with edge guidance using: https://github.com/lllyasviel/ControlNet/discussions/12.
Model Details
| Property |
Details |
| Developed by |
Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan and Humphrey Shi |
| Model Type |
Dreambooth text - to - image and text - to - video generation model with edge control for text2video zero |
| Language(s) |
English |
| License |
The CreativeML OpenRAIL M license |
| Model Description |
This is a model for text2video zero with edge guidance and avatar style. It can also be used with ControlNet in a text - to - image setup with edge guidance. |
| DreamBoth Keyword |
avatar style |
| Resources for more information |
GitHub, Paper, CIVITAI |
Original Weights
The Dreambooth weights for the Avatar style were taken from CIVITAI.
Model Details
| Property |
Details |
| Developed by |
Quiet_Joker (Username listed on CIVITAI) |
| Model Type |
Dreambooth text - to - image generation model |
| Language(s) |
English |
| License |
The CreativeML OpenRAIL M license |
| Model Description |
This is a model that was created using DreamBooth to generate images with avatar style, based on text prompts. |
| DreamBoth Keyword |
avatar style |
| Resources for more information |
CIVITAI |
Biases content acknowledgement
â ī¸ Important Note
Be aware that Text2Video - Zero may output content that reinforces or exacerbates societal biases, as well as realistic faces, pornography, and violence. Text2Video - Zero in this demo is for research purposes only.
đ License
The model is under The CreativeML OpenRAIL M license.
đ Citation
@article{text2video-zero,
title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators},
author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey},
journal={arXiv preprint arXiv:2303.13439},
year={2023}
}
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Transformers