%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
EN
Wan2.1 Fun V1.1 14B Control
A 1.3B parameter text-to-video model developed by Alibaba Cloud PAI team, supporting multi-resolution training and first-last frame prediction
Downloads 55
Release Time : 4/24/2025
Model Overview
This model is a diffusion-based (diffusers) text-to-video generation system capable of producing high-quality video content from text descriptions. It supports multiple output resolutions and features first-last frame prediction to enhance video coherence.
Model Features
Multi-resolution support
Supports video generation at various resolutions including 512/768/1024
First-last frame prediction
Special technique to enhance video coherence
Multi-language compatibility
Supports Chinese and English text input
Control condition support
Can generate videos with control conditions like Canny, Depth, Pose
Model Capabilities
Text-to-video generation
Multi-resolution video generation
Condition-controlled video generation
First-last frame prediction
Use Cases
Creative content generation
Short video creation
Automatically generate creative short videos from text descriptions
81 frames, 16FPS smooth video
Advertisement production
Quickly generate product showcase videos
Professional-grade videos in multiple resolutions
Education & entertainment
Educational video generation
Transform textbook content into vivid videos
Supports Chinese and English teaching materials
๐ Wan-Fun
๐ Welcome! This project focuses on text - to - video generation, offering various models and functionalities for creating high - quality videos.
๐ Quick Start
1. Cloud Usage: AliyunDSW/Docker
a. Via Alibaba Cloud DSW
DSW provides free GPU usage time. Users can apply for it once, and the application is valid for 3 months.
Alibaba Cloud offers free GPU time on Freetier. Obtain it and use it in Alibaba Cloud PAI - DSW. You can start CogVideoX - Fun within 5 minutes.
b. Via ComfyUI
Our ComfyUI interface is as follows. For details, check ComfyUI README.
c. Via Docker
If you use Docker, make sure the graphics card driver and CUDA environment are correctly installed on your machine, and then execute the following commands:
# pull image
docker pull mybigpai-public-registry.cn-beijing.cr.aliyuncs.com/easycv/torch_cuda:cogvideox_fun
# enter image
docker run -it -p 7860:7860 --network host --gpus all --security-opt seccomp:unconfined --shm-size 200g mybigpai-public-registry.cn-beijing.cr.aliyuncs.com/easycv/torch_cuda:cogvideox_fun
# clone code
git clone https://github.com/aigc-apps/VideoX-Fun.git
# enter VideoX-Fun's dir
cd VideoX-Fun
# download weights
mkdir models/Diffusion_Transformer
mkdir models/Personalized_Model
# Please use the hugginface link or modelscope link to download the model.
# CogVideoX-Fun
# https://huggingface.co/alibaba-pai/CogVideoX-Fun-V1.1-5b-InP
# https://modelscope.cn/models/PAI/CogVideoX-Fun-V1.1-5b-InP
# Wan
# https://huggingface.co/alibaba-pai/Wan2.1-Fun-V1.1-14B-InP
# https://modelscope.cn/models/PAI/Wan2.1-Fun-V1.1-14B-InP
2. Local Installation: Environment Check/Download/Installation
a. Environment Check
We have verified that this library can be executed in the following environments:
Details for Windows:
- Operating System: Windows 10
- Python: python3.10 & python3.11
- PyTorch: torch2.2.0
- CUDA: 11.8 & 12.1
- CUDNN: 8+
- GPU: Nvidia - 3060 12G & Nvidia - 3090 24G
Details for Linux:
- Operating System: Ubuntu 20.04, CentOS
- Python: python3.10 & python3.11
- PyTorch: torch2.2.0
- CUDA: 11.8 & 12.1
- CUDNN: 8+
- GPU: Nvidia - V100 16G & Nvidia - A10 24G & Nvidia - A100 40G & Nvidia - A100 80G
You need approximately 60GB of available disk space. Please check!
b. Weight Placement
It is recommended to place the weights according to the specified paths:
Via ComfyUI:
Place the models in the weight folder ComfyUI/models/Fun_Models/ of ComfyUI:
๐ฆ ComfyUI/
โโโ ๐ models/
โ โโโ ๐ Fun_Models/
โ โโโ ๐ CogVideoX-Fun-V1.1-2b-InP/
โ โโโ ๐ CogVideoX-Fun-V1.1-5b-InP/
โ โโโ ๐ Wan2.1-Fun-V1.1-14B-InP
โ โโโ ๐ Wan2.1-Fun-V1.1-1.3B-InP/
Running your own Python files or UI interface:
๐ฆ models/
โโโ ๐ Diffusion_Transformer/
โ โโโ ๐ CogVideoX-Fun-V1.1-2b-InP/
โ โโโ ๐ CogVideoX-Fun-V1.1-5b-InP/
โ โโโ ๐ Wan2.1-Fun-V1.1-14B-InP
โ โโโ ๐ Wan2.1-Fun-V1.1-1.3B-InP/
โโโ ๐ Personalized_Model/
โ โโโ your trained trainformer model / your trained lora model (for UI load)
โจ Features
Model Addresses
V1.1:
| Name | Storage Space | Hugging Face | Model Scope | Description |
|---|---|---|---|---|
| Wan2.1 - Fun - V1.1 - 1.3B - InP | 19.0 GB | ๐คLink | ๐Link | Weights for text - to - video generation of Wan2.1 - Fun - V1.1 - 1.3B, trained at multiple resolutions, supporting prediction of start and end frames. |
| Wan2.1 - Fun - V1.1 - 14B - InP | 47.0 GB | ๐คLink | ๐Link | Weights for text - to - video generation of Wan2.1 - Fun - V1.1 - 14B, trained at multiple resolutions, supporting prediction of start and end frames. |
| Wan2.1 - Fun - V1.1 - 1.3B - Control | 19.0 GB | ๐คLink | ๐Link | Video control weights of Wan2.1 - Fun - V1.1 - 1.3B, supporting different control conditions such as Canny, Depth, Pose, MLSD, etc., supporting control with reference images + control conditions, and supporting trajectory control. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, and supports multi - language prediction. |
| Wan2.1 - Fun - V1.1 - 14B - Control | 47.0 GB | ๐คLink | ๐Link | Video control weights of Wan2.1 - Fun - V1.1 - 14B, supporting different control conditions such as Canny, Depth, Pose, MLSD, etc., supporting control with reference images + control conditions, and supporting trajectory control. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, and supports multi - language prediction. |
| Wan2.1 - Fun - V1.1 - 1.3B - Control - Camera | 19.0 GB | ๐คLink | ๐Link | Camera lens control weights of Wan2.1 - Fun - V1.1 - 1.3B. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, and supports multi - language prediction. |
| Wan2.1 - Fun - V1.1 - 14B - Control | 47.0 GB | ๐คLink | ๐Link | Camera lens control weights of Wan2.1 - Fun - V1.1 - 14B. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, and supports multi - language prediction. |
V1.0:
| Name | Storage Space | Hugging Face | Model Scope | Description |
|---|---|---|---|---|
| Wan2.1 - Fun - 1.3B - InP | 19.0 GB | ๐คLink | ๐Link | Weights for text - to - video generation of Wan2.1 - Fun - 1.3B, trained at multiple resolutions, supporting prediction of start and end frames. |
| Wan2.1 - Fun - 14B - InP | 47.0 GB | ๐คLink | ๐Link | Weights for text - to - video generation of Wan2.1 - Fun - 14B, trained at multiple resolutions, supporting prediction of start and end frames. |
| Wan2.1 - Fun - 1.3B - Control | 19.0 GB | ๐คLink | ๐Link | Video control weights of Wan2.1 - Fun - 1.3B, supporting different control conditions such as Canny, Depth, Pose, MLSD, etc., and supporting trajectory control. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, and supports multi - language prediction. |
| Wan2.1 - Fun - 14B - Control | 47.0 GB | ๐คLink | ๐Link | Video control weights of Wan2.1 - Fun - 14B, supporting different control conditions such as Canny, Depth, Pose, MLSD, etc., and supporting trajectory control. Supports video prediction at multiple resolutions (512, 768, 1024), trained with 81 frames at 16 frames per second, and supports multi - language prediction. |
Video Works
Wan2.1 - Fun - V1.1 - 14B - InP && Wan2.1 - Fun - V1.1 - 1.3B - InP
Wan2.1 - Fun - V1.1 - 14B - Control && Wan2.1 - Fun - V1.1 - 1.3B - Control
Generic Control Video + Reference Image:
| Reference Image | Control Video | Wan2.1 - Fun - V1.1 - 14B - Control | Wan2.1 - Fun - V1.1 - 1.3B - Control |
|
|
|||
Generic Control Video (Canny, Pose, Depth, etc.) and Trajectory Control:
Wan2.1 - Fun - V1.1 - 14B - Control - Camera && Wan2.1 - Fun - V1.1 - 1.3B - Control - Camera
| Pan Up | Pan Left | Pan Right |
| Pan Down | Pan Up + Pan Left | Pan Up + Pan Right |
๐ป Usage Examples
1. Generation
a. Video Memory Saving Scheme
Since the parameters of Wan2.1 are very large, we need to consider a video memory saving scheme to adapt to consumer - grade graphics cards. We provide a GPU_memory_mode for each prediction file, which can be selected from model_cpu_offload, model_cpu_offload_and_qfloat8, and sequential_cpu_offload. This scheme is also applicable to the generation of CogVideoX - Fun.
model_cpu_offloadmeans the entire model will be moved to the CPU after use, which can save some video memory.model_cpu_offload_and_qfloat8means the entire model will be moved to the CPU after use, and the transformer model is quantized to float8, which can save more video memory.sequential_cpu_offloadmeans each layer of the model will be moved to the CPU after use. It is slower but saves a large amount of video memory.
qfloat8 will partially reduce the performance of the model but save more video memory. If the video memory is sufficient, it is recommended to use model_cpu_offload.
b. Via ComfyUI
Check ComfyUI README for details.
c. Running Python Files
- Step 1: Download the corresponding [weights]
๐ License
The project is licensed under the Apache - 2.0 license.
Xclip Base Patch32
MIT
X-CLIP is an extended version of CLIP for general video-language understanding, trained on (video, text) pairs via contrastive learning, suitable for tasks like video classification and video-text retrieval.
Text-to-Video
Transformers English
Transformers EnglishX
microsoft
309.80k
84
LTX Video
Other
The first DiT-based video generation model capable of real-time generation of high-quality videos, supporting two scenarios: text-to-video and image + text-to-video.
Text-to-Video English
L
Lightricks
165.42k
1,174
Wan2.1 14B VACE GGUF
Apache-2.0
The GGUF format version of the Wan2.1-VACE-14B model, mainly used for text-to-video generation tasks.
Text-to-Video
W
QuantStack
146.36k
139
Animatediff Lightning
Openrail
Ultra-fast text-to-video model, generating videos over ten times faster than the original AnimateDiff
Text-to-Video
A
ByteDance
144.00k
925
V Express
V-Express is an audio and facial keypoint condition-based video generation model capable of converting audio input into dynamic video output.
Text-to-Video English
V
tk93
118.36k
85
Cogvideox 5b
Other
CogVideoX is the open-source version of the video generation model derived from Qingying, providing high-quality video generation capabilities.
Text-to-Video English
C
THUDM
92.32k
611
Llava NeXT Video 7B Hf
LLaVA-NeXT-Video is an open-source multimodal chatbot that achieves excellent video understanding capabilities through mixed training on video and image data, reaching SOTA level among open-source models on the VideoMME benchmark.
Text-to-Video Transformers English
Transformers EnglishL
llava-hf
65.95k
88
Wan2.1 T2V 14B Diffusers
Apache-2.0
Wan2.1 is a comprehensive open-source video foundation model designed to push the boundaries of video generation, supporting tasks such as text-to-video in Chinese and English, image-to-video, and more.
Text-to-Video Supports Multiple Languages
W
Wan-AI
48.65k
24
Wan2.1 T2V 1.3B Diffusers
Apache-2.0
Wan 2.1 is a comprehensive open-source video foundation model featuring top-tier performance, consumer-grade GPU support, multi-task capabilities, visual-text generation, and efficient video VAE.
Text-to-Video Supports Multiple Languages
W
Wan-AI
45.29k
38
Wan2.1 T2V 14B
Apache-2.0
Wan 2.1 is a comprehensive open-source video foundation model capable of multiple tasks including text-to-video, image-to-video, video editing, text-to-image, and video-to-audio generation, with support for bilingual Chinese-English text input.
Text-to-Video Supports Multiple Languages
W
Wan-AI
44.88k
1,238
Featured Recommended AI Models
ยฉ 2025AIbase