Pika Dissolve V0

🚀 PIKA_DISSOLVE 微調模型項目

本項目是基於 THUDM/CogVideoX - 5b 模型，在 modal - labs/dissolve 數據集上進行微調的成果。它能夠根據文本描述生成特定的視頻效果，例如物體溶解的動態視頻。

🚀 快速開始

代碼倉庫

項目代碼可在 此處 獲取。

推理代碼示例

以下是使用微調模型進行推理的代碼示例：

from diffusers import CogVideoXTransformer3DModel, DiffusionPipeline 
from diffusers.utils import export_to_video
import torch 

transformer = CogVideoXTransformer3DModel.from_pretrained(
    "sayakpaul/pika-dissolve-v0", torch_dtype=torch.bfloat16
)
pipeline = DiffusionPipeline.from_pretrained(
    "THUDM/CogVideoX-5b", transformer=transformer, torch_dtype=torch.bfloat16
).to("cuda")

prompt = """
PIKA_DISSOLVE A slender glass vase, brimming with tiny white pebbles, stands centered on a polished ebony dais. Without warning, the glass begins to dissolve from the edges inward. Wisps of translucent dust swirl upward in an elegant spiral, illuminating each pebble as they drop onto the dais. The gently drifting dust eventually settles, leaving only the scattered stones and faint traces of shimmering powder on the stage.
"""
negative_prompt = "inconsistent motion, blurry motion, worse quality, degenerate outputs, deformed outputs"

video = pipeline(
    prompt=prompt, 
    negative_prompt=negative_prompt, 
    num_frames=81, 
    height=512,
    width=768,
    num_inference_steps=50
).frames[0]
export_to_video(video, "output_vase.mp4", fps=25)

📦 模型與數據信息

📄 許可證

本項目使用的許可證為 other。

💻 使用示例

基礎用法

上述推理代碼即為基礎的使用示例，通過定義輸入的文本描述 prompt 和負面提示 negative_prompt，可以生成相應的視頻。

示例輸出展示

以下是一些不同文本描述對應的視頻輸出示例：

• 文本描述：PIKA_DISSOLVE A meticulously detailed, tea cup, sits centrally on a dark brown circular pedestal. The cup, seemingly made of clay, begins to dissolve from the bottom up. The disintegration process is rapid but not explosive, with a cloud of fine, light tan dust forming and rising in a swirling, almost ethereal column that expands outwards before slowly descending. The dust particles are individually visible as they float, and the overall effect is one of delicate disintegration rather than shattering. Finally, only the empty pedestal and the intricately patterned marble floor remain. 輸出視頻：點擊查看
• 文本描述：PIKA_DISSOLVE Resting quietly atop an ancient stone altar, a delicately carved wooden mask starts to crumble from its outer edges. The intricate patterns crack and give way, releasing a fine, smoke - like plume of mahogany - hued particles that dance upwards, then disperse gradually into the hushed atmosphere. As the dust descends, the once captivating mask is reduced to an outline on the weathered altar. 輸出視頻：點擊查看
• 文本描述：PIKA_DISSOLVE A slender glass vase, brimming with tiny white pebbles, stands centered on a polished ebony dais. Without warning, the glass begins to dissolve from the edges inward. Wisps of translucent dust swirl upward in an elegant spiral, illuminating each pebble as they drop onto the dais. The gently drifting dust eventually settles, leaving only the scattered stones and faint traces of shimmering powder on the stage. 輸出視頻：點擊查看
• 文本描述：PIKA_DISSOLVE On a narrow marble ledge, a gracefully folded paper crane rests, its surface marked by delicate ink lines. It starts to fragment from the tail feathers outward, releasing a cloud of feather - light pulp fibers. Suspended for a moment in a magical swirl, the fibers drift back down, cloaking the ledge in a near - transparent veil of white. Then the ledge stands empty, the crane’s faint silhouette lingering in memory. 輸出視頻：點擊查看

🏷️ 標籤

本項目相關的標籤有：text - to - video、diffusers - training、diffusers、cogvideox、cogvideox - diffusers、template:sd - lora。