đ Text2Video-Zero Model Card - ControlNet Canny Aracane Style
Text2Video-Zero is a zero-shot text-to-video generator. It can perform multiple tasks such as zero-shot text-to-video generation, Video Instruct Pix2Pix (instruction-guided video editing), text and pose conditional video generation, text and canny-edge conditional video generation, and text, canny-edge and dreambooth conditional video generation. This model provides DreamBooth weights for the Arcane style to be used with edge guidance in text2video zero.
đ Quick Start
The Text2Video-Zero can perform various video generation and editing tasks. Our code is compatible with any StableDiffusion base model. For more information, please refer to our paper and our demo: 
⨠Features
- Multiple Generation Modes: Capable of zero-shot text-to-video generation, instruction-guided video editing, and conditional video generation based on text, pose, canny-edge, etc.
- Arcane Style Support: Provides DreamBooth weights for the
Arcane style with edge guidance.
- Compatibility: Works with any StableDiffusion base model.
đĻ Installation
The README does not provide specific installation steps, so this section is skipped.
đģ Usage Examples
The README does not provide code examples, so this section is skipped.
đ Documentation
Weights for Text2Video-Zero
We converted the original weights into diffusers and made them usable for ControlNet with edge guidance using: https://github.com/lllyasviel/ControlNet/discussions/12.
Model Details
| Property |
Details |
| Developed by |
Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan and Humphrey Shi |
| Model Type |
Dreambooth text-to-image and text-to-video generation model with edge control for text2video zero |
| Language(s) |
English |
| License |
The CreativeML OpenRAIL M license. |
| Model Description |
This is a model for text2video zero with edge guidance and arcane style. It can be used also with ControlNet in a text-to-image setup with edge guidance. |
| DreamBoth Keyword |
arcane style |
| Resources for more information |
GitHub, Paper, CIVITAI. |
| Cite as |
@article{text2video-zero,
title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators},
author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey},
journal={arXiv preprint arXiv:2303.13439},
year={2023}
} |
Original Weights
The Dreambooth weights for the Arcane style were taken from CIVITAI.
Model Details
| Property |
Details |
| Developed by |
Quiet_Joker (Username listed on CIVITAI) |
| Model Type |
Dreambooth text-to-image generation model |
| Language(s) |
English |
| License |
The CreativeML OpenRAIL M license. |
| Model Description |
This is a model that was created using DreamBooth to generate images with Arcane style, based on text prompts. |
| DreamBoth Keyword |
arcane style |
| Resources for more information |
CIVITAI. |
Biases content acknowledgement
â ī¸ Important Note
Beware that Text2Video-Zero may output content that reinforces or exacerbates societal biases, as well as realistic faces, pornography, and violence. Text2Video-Zero in this demo is meant only for research purposes.
đ License
The model is licensed under The CreativeML OpenRAIL M license.
đ Citation
@article{text2video-zero,
title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators},
author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey},
journal={arXiv preprint arXiv:2303.13439},
year={2023}
}
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Transformers