đ Text2Video-Zero Model Card - ControlNet Canny Anime Style
Text2Video-Zero is a zero-shot text-to-video generator. It can perform various tasks such as zero-shot text-to-video generation, Video Instruct Pix2Pix (instruction-guided video editing), text and pose conditional video generation, text and canny-edge conditional video generation, and text, canny-edge and dreambooth conditional video generation. This model provides DreamBooth weights for the Anime style to be used with edge guidance in text2video zero.
đ Quick Start
Text2Video-Zero is a powerful zero-shot text-to-video generator. It can perform multiple tasks, including zero-shot text-to-video generation, Video Instruct Pix2Pix (instruction-guided video editing), text and pose conditional video generation, text and canny-edge conditional video generation, and text, canny-edge and dreambooth conditional video generation. For more information, please refer to our paper and our demo: 
. Our code is compatible with any StableDiffusion base model.
This model offers DreamBooth weights for the Anime style to be used with edge guidance (using ControlNet) in text2video zero.
⨠Features
- Multiple Generation Modes: Supports zero-shot text-to-video generation, instruction-guided video editing, and conditional video generation based on text, pose, canny-edge, and dreambooth.
- Anime Style Support: Provides DreamBooth weights for the Anime style with edge guidance.
- Compatibility: Works with any StableDiffusion base model.
đ Documentation
Weights for Text2Video-Zero
We converted the original weights into diffusers and made them usable for ControlNet with edge guidance using: https://github.com/lllyasviel/ControlNet/discussions/12.
Model Details
| Property |
Details |
| Developed by |
Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan and Humphrey Shi |
| Model Type |
Dreambooth text-to-image and text-to-video generation model with edge control for text2video zero |
| Language(s) |
English |
| License |
The CreativeML OpenRAIL M license |
| Model Description |
This is a model for text2video zero with edge guidance and anime style. It can also be used with ControlNet in a text-to-image setup with edge guidance. |
| DreamBoth Keyword |
anime style |
| Resources for more information |
GitHub, Paper, CIVITAI |
Original Weights
The Dreambooth weights for the Anime style were taken from CIVITAI.
Model Details
| Property |
Details |
| Developed by |
Quiet_Joker (Username listed on CIVITAI) |
| Model Type |
Dreambooth text-to-image generation model |
| Language(s) |
English |
| License |
The CreativeML OpenRAIL M license |
| Model Description |
This is a model that was created using DreamBooth to generate images with Anime style, based on text prompts. |
| DreamBoth Keyword |
anime style |
| Resources for more information |
CIVITAI |
Biases content acknowledgement
â ī¸ Important Note
Beware that Text2Video-Zero may output content that reinforces or exacerbates societal biases, as well as realistic faces, pornography, and violence. Text2Video-Zero in this demo is meant only for research purposes.
đ License
This model is licensed under The CreativeML OpenRAIL M license.
đ Citation
@article{text2video-zero,
title={Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators},
author={Khachatryan, Levon and Movsisyan, Andranik and Tadevosyan, Vahram and Henschel, Roberto and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey},
journal={arXiv preprint arXiv:2303.13439},
year={2023}
}
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Transformers