%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
EN
Wan2.1 Fun 1.3B InP
Wan2.1-Fun-1.3B is a text-to-video generation model developed by Alibaba PAI team, supporting multi-resolution training and first/last frame prediction.
Downloads 6,753
Release Time : 3/17/2025
Model Overview
This model specializes in text-to-video generation tasks, capable of producing high-quality video content from text descriptions, supporting multiple resolutions and multilingual input.
Model Features
Multi-resolution support
Supports video generation at various resolutions (512/768/1024 etc.) to meet different application requirements.
First/last frame prediction
Capable of predicting the first and last frames of videos to enhance coherence and quality of generated content.
Multilingual input
Supports English and Chinese text input to accommodate users of different languages.
Model Capabilities
Text-to-video generation
Multi-resolution video generation
First/last frame prediction
Multilingual support
Use Cases
Creative content generation
Short video creation
Automatically generates creative short video content based on text descriptions.
Produces high-quality, coherent short video clips.
Advertisement production
Quickly generates product promotional videos.
Reduces production time and costs.
Education and training
Educational video generation
Automatically generates supporting videos based on teaching content.
Improves teaching efficiency and enriches instructional formats.
๐ Wan-Fun
Wan-Fun is a text-to-video model that enables video generation from text or images. It supports multiple resolutions and various control conditions, offering a powerful solution for video creation.
๐ Welcome!
๐ Quick Start
1. Cloud Usage: AliyunDSW/Docker
a. Through Alibaba Cloud DSW
DSW offers free GPU hours, which users can apply for once and are valid for 3 months after application.
Alibaba Cloud provides free GPU hours on Freetier. Obtain them and use them in Alibaba Cloud PAI-DSW to start CogVideoX-Fun within 5 minutes.
b. Through ComfyUI
Our ComfyUI interface is as follows. For details, check ComfyUI README.
c. Through Docker
If using Docker, ensure that the graphics card driver and CUDA environment are correctly installed on the machine, and then execute the following commands in sequence:
# pull image
docker pull mybigpai-public-registry.cn-beijing.cr.aliyuncs.com/easycv/torch_cuda:cogvideox_fun
# enter image
docker run -it -p 7860:7860 --network host --gpus all --security-opt seccomp:unconfined --shm-size 200g mybigpai-public-registry.cn-beijing.cr.aliyuncs.com/easycv/torch_cuda:cogvideox_fun
# clone code
git clone https://github.com/aigc-apps/CogVideoX-Fun.git
# enter CogVideoX-Fun's dir
cd CogVideoX-Fun
# download weights
mkdir models/Diffusion_Transformer
mkdir models/Personalized_Model
# Please use the hugginface link or modelscope link to download the model.
# CogVideoX-Fun
# https://huggingface.co/alibaba-pai/CogVideoX-Fun-V1.1-5b-InP
# https://modelscope.cn/models/PAI/CogVideoX-Fun-V1.1-5b-InP
# Wan
# https://huggingface.co/alibaba-pai/Wan2.1-Fun-14B-InP
# https://modelscope.cn/models/PAI/Wan2.1-Fun-14B-InP
2. Local Installation: Environment Check/Download/Installation
a. Environment Check
We have verified that this library can run in the following environments:
Details for Windows:
- Operating System: Windows 10
- Python: python3.10 & python3.11
- PyTorch: torch2.2.0
- CUDA: 11.8 & 12.1
- CUDNN: 8+
- GPU: Nvidia-3060 12G & Nvidia-3090 24G
Details for Linux:
- Operating System: Ubuntu 20.04, CentOS
- Python: python3.10 & python3.11
- PyTorch: torch2.2.0
- CUDA: 11.8 & 12.1
- CUDNN: 8+
- GPU: Nvidia-V100 16G & Nvidia-A10 24G & Nvidia-A100 40G & Nvidia-A100 80G
We need approximately 60GB of available disk space. Please check!
b. Weight Placement
It is recommended to place the weights in the specified paths:
๐ฆ models/
โโโ ๐ Diffusion_Transformer/
โ โโโ ๐ CogVideoX-Fun-V1.1-2b-InP/
โ โโโ ๐ CogVideoX-Fun-V1.1-5b-InP/
โ โโโ ๐ Wan2.1-Fun-14B-InP
โ โโโ ๐ Wan2.1-Fun-1.3B-InP/
โโโ ๐ Personalized_Model/
โ โโโ your trained trainformer model / your trained lora model (for UI load)
โจ Features
Model Address
V1.0:
| Name | Storage Space | Hugging Face | Model Scope | Description |
|---|---|---|---|---|
| Wan2.1-Fun-1.3B-InP | 19.0 GB | ๐คLink | ๐Link | Wan2.1-Fun-1.3B text-to-video weights, trained at multiple resolutions, supporting first and last frame prediction. |
| Wan2.1-Fun-14B-InP | 47.0 GB | ๐คLink | ๐Link | Wan2.1-Fun-14B text-to-video weights, trained at multiple resolutions, supporting first and last frame prediction. |
| Wan2.1-Fun-1.3B-Control | 19.0 GB | ๐คLink | ๐Link | Wan2.1-Fun-1.3B video control weights, supporting different control conditions such as Canny, Depth, Pose, MLSD, etc., and trajectory control. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, supporting multi-language prediction. |
| Wan2.1-Fun-14B-Control | 47.0 GB | ๐คLink | ๐Link | Wan2.1-Fun-14B video control weights, supporting different control conditions such as Canny, Depth, Pose, MLSD, etc., and trajectory control. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, supporting multi-language prediction. |
Video Works
Wan2.1-Fun-14B-InP && Wan2.1-Fun-1.3B-InP
Wan2.1-Fun-14B-Control && Wan2.1-Fun-1.3B-Control
๐ป Usage Examples
1. Generation
a. Memory Saving Scheme
Due to the large number of parameters in Wan2.1, we need to consider a memory-saving scheme to adapt to consumer-grade graphics cards. We provide a GPU_memory_mode for each prediction file, which can be selected from model_cpu_offload, model_cpu_offload_and_qfloat8, and sequential_cpu_offload. This scheme also applies to the generation of CogVideoX-Fun.
- model_cpu_offload means the entire model will be moved to the CPU after use, saving some video memory.
- model_cpu_offload_and_qfloat8 means the entire model will be moved to the CPU after use, and the transformer model is quantized to float8, saving more video memory.
- sequential_cpu_offload means each layer of the model will be moved to the CPU after use. It is slower but saves a large amount of video memory.
qfloat8 will partially reduce the model's performance but save more video memory. If the video memory is sufficient, it is recommended to use model_cpu_offload.
b. Through ComfyUI
For details, check ComfyUI README.
c. Running Python Files
- Step 1: Download the corresponding weights and place them in the models folder.
- Step 2: Use different files for prediction based on different weights and prediction targets. Currently, this library supports CogVideoX-Fun, Wan2.1, and Wan2.1-Fun, which are distinguished by folder names in the examples folder. Different models support different functions, so please distinguish them according to the specific situation. Take CogVideoX-Fun as an example.
- Text-to-Video:
- Modify prompt, neg_prompt, guidance_scale, and seed in the examples/cogvideox_fun/predict_t2v.py file.
- Then run the examples/cogvideox_fun/predict_t2v.py file and wait for the generation result. The result will be saved in the samples/cogvideox-fun-videos folder.
- Image-to-Video:
- Modify validation_image_start, validation_image_end, prompt, neg_prompt, guidance_scale, and seed in the examples/cogvideox_fun/predict_i2v.py file.
- validation_image_start is the starting image of the video, and validation_image_end is the ending image of the video.
- Then run the examples/cogvideox_fun/predict_i2v.py file and wait for the generation result. The result will be saved in the samples/cogvideox-fun-videos_i2v folder.
- Video-to-Video:
- Modify validation_video, validation_image_end, prompt, neg_prompt, guidance_scale, and seed in the examples/cogvideox_fun/predict_v2v.py file.
- validation_video is the reference video for video-to-video generation. You can use the following video for the demo: Demo Video
- Then run the examples/cogvideox_fun/predict_v2v.py file and wait for the generation result. The result will be saved in the samples/cogvideox-fun-videos_v2v folder.
- Normal Control-to-Video (Canny, Pose, Depth, etc.):
- Modify control_video, validation_image_end, prompt, neg_prompt, guidance_scale, and seed in the examples/cogvideox_fun/predict_v2v_control.py file.
- control_video is the control video for control-to-video generation, which is a video extracted by operators such as Canny, Pose, and Depth. You can use the following video for the demo: Demo Video
- Then run the examples/cogvideox_fun/predict_v2v_control.py file and wait for the generation result. The result will be saved in the samples/cogvideox-fun-videos_v2v_control folder.
- Text-to-Video:
- Step 3: If you want to combine other backbones and LoRA trained by yourself, modify the lora_path in examples/{model_name}/predict_t2v.py and examples/{model_name}/predict_i2v.py according to the situation.
d. Through the UI Interface
The webui supports text-to-video, image-to-video, video-to-video, and normal control-to-video (Canny, Pose, Depth, etc.). Currently, this library supports CogVideoX-Fun, Wan2.1, and Wan2.1-Fun, which are distinguished by folder names in the examples folder. Different models support different functions, so please distinguish them according to the specific situation. Take CogVideoX-Fun as an example.
- Step 1: Download the corresponding weights and place them in the models folder.
- Step 2: Run the examples/cogvideox_fun/app.py file and enter the Gradio page.
- Step 3: Select the generation model on the page, fill in prompt, neg_prompt, guidance_scale, and seed, etc., click Generate, and wait for the generation result. The result will be saved in the sample folder.
๐ Documentation
References
- CogVideo: https://github.com/THUDM/CogVideo/
- EasyAnimate: https://github.com/aigc-apps/EasyAnimate
- Wan2.1: https://github.com/Wan-Video/Wan2.1/
๐ License
This project is licensed under the Apache License (Version 2.0).
Xclip Base Patch32
MIT
X-CLIP is an extended version of CLIP for general video-language understanding, trained on (video, text) pairs via contrastive learning, suitable for tasks like video classification and video-text retrieval.
Text-to-Video
Transformers English
Transformers EnglishX
microsoft
309.80k
84
LTX Video
Other
The first DiT-based video generation model capable of real-time generation of high-quality videos, supporting two scenarios: text-to-video and image + text-to-video.
Text-to-Video English
L
Lightricks
165.42k
1,174
Wan2.1 14B VACE GGUF
Apache-2.0
The GGUF format version of the Wan2.1-VACE-14B model, mainly used for text-to-video generation tasks.
Text-to-Video
W
QuantStack
146.36k
139
Animatediff Lightning
Openrail
Ultra-fast text-to-video model, generating videos over ten times faster than the original AnimateDiff
Text-to-Video
A
ByteDance
144.00k
925
V Express
V-Express is an audio and facial keypoint condition-based video generation model capable of converting audio input into dynamic video output.
Text-to-Video English
V
tk93
118.36k
85
Cogvideox 5b
Other
CogVideoX is the open-source version of the video generation model derived from Qingying, providing high-quality video generation capabilities.
Text-to-Video English
C
THUDM
92.32k
611
Llava NeXT Video 7B Hf
LLaVA-NeXT-Video is an open-source multimodal chatbot that achieves excellent video understanding capabilities through mixed training on video and image data, reaching SOTA level among open-source models on the VideoMME benchmark.
Text-to-Video Transformers English
Transformers EnglishL
llava-hf
65.95k
88
Wan2.1 T2V 14B Diffusers
Apache-2.0
Wan2.1 is a comprehensive open-source video foundation model designed to push the boundaries of video generation, supporting tasks such as text-to-video in Chinese and English, image-to-video, and more.
Text-to-Video Supports Multiple Languages
W
Wan-AI
48.65k
24
Wan2.1 T2V 1.3B Diffusers
Apache-2.0
Wan 2.1 is a comprehensive open-source video foundation model featuring top-tier performance, consumer-grade GPU support, multi-task capabilities, visual-text generation, and efficient video VAE.
Text-to-Video Supports Multiple Languages
W
Wan-AI
45.29k
38
Wan2.1 T2V 14B
Apache-2.0
Wan 2.1 is a comprehensive open-source video foundation model capable of multiple tasks including text-to-video, image-to-video, video editing, text-to-image, and video-to-audio generation, with support for bilingual Chinese-English text input.
Text-to-Video Supports Multiple Languages
W
Wan-AI
44.88k
1,238
Featured Recommended AI Models
ยฉ 2025AIbase