🚀 AnyStory：文本到圖像生成中實現統一的單主體和多主體個性化

AnyStory是一種用於個性化主體生成的統一方法。它不僅能為單主體實現高保真的個性化，還能為多主體實現這一目標，且不會犧牲主體的保真度。

這個模型倉庫是關於 AnyStory 的。

✨ 主要特性

AnyStory是一種統一的個性化主體生成方法，具備以下特點：

• 能夠為單主體和多主體實現高保真的個性化。
• 在多主體個性化過程中，不會犧牲主體的保真度。

📦 安裝指南

文檔中未提及具體安裝步驟，暫不提供。

💻 使用示例

基礎用法

import torch
from PIL import Image

from huggingface_hub import hf_hub_download
from anystory.generate import AnyStoryFluxPipeline

anystory_path = hf_hub_download(repo_id="Junjie96/AnyStory", filename="anystory_flux.bin")
story_pipe = AnyStoryFluxPipeline(
    hf_flux_pipeline_path="black-forest-labs/FLUX.1-dev",
    hf_flux_redux_path="black-forest-labs/FLUX.1-Redux-dev",
    anystory_path=anystory_path,
    device="cuda",
    torch_dtype=torch.bfloat16
)
# you can add lora here
# story_pipe.flux_pipeline.load_lora_weights(lora_path, adapter_name="...")

# single-subject
subject_image = Image.open("assets/examples/1.webp").convert("RGB")
subject_mask = Image.open("assets/examples/1_mask.webp").convert("L")
prompt = "Cartoon style. A sheep is riding a skateboard and gliding through the city," \
         " holding a wooden sign that says \"hello\"."
image = story_pipe.generate(prompt=prompt, images=[subject_image], masks=[subject_mask], seed=2025,
                            num_inference_steps=25, height=512, width=512,
                            guidance_scale=3.5)
image.save("output_1.png")

# multi-subject
subject_image_1 = Image.open("assets/examples/6_1.webp").convert("RGB")
subject_mask_1 = Image.open("assets/examples/6_1_mask.webp").convert("L")
subject_image_2 = Image.open("assets/examples/6_2.webp").convert("RGB")
subject_mask_2 = Image.open("assets/examples/6_2_mask.webp").convert("L")
prompt = "Two men are sitting by a wooden table, which is laden with delicious food and a pot of wine. " \
         "One of the men holds a wine glass, drinking heartily with a bold expression; " \
         "the other smiles as he pours wine for his companion, both of them engaged in cheerful conversation. " \
         "In the background is an ancient pavilion surrounded by emerald bamboo groves, with sunlight filtering " \
         "through the leaves to cast dappled shadows."

image = story_pipe.generate(prompt=prompt,
                            images=[subject_image_1, subject_image_2],
                            masks=[subject_mask_1, subject_mask_2],
                            seed=2025,
                            enable_router=True, ref_start_at=0.09,
                            num_inference_steps=25, height=512, width=512,
                            guidance_scale=3.5)
image.save("output_2.png")

故事板生成

import json
from storyboard import StoryboardPipeline

storyboard_pipe = StoryboardPipeline()

storyboard_pipe.new_story()
script_dict = json.load(open("assets/scripts/013420.json"))
print(script_dict)
results = storyboard_pipe(script_dict, style_name="Comic book")
for key, result in results.items():
    result.save(f"output_1_{key}.png")

# 狮子王辛巴成长
storyboard_pipe.new_story()
script_dict = json.load(open("assets/scripts/014933.json"))
print(script_dict)
results = storyboard_pipe(script_dict, style_name="Japanese Anime")
for key, result in results.items():
    result.save(f"output_2_{key}.png")

示例輸出：

📚 詳細文檔

應用場景

• 智能創作AI故事圖片：可結合 Qwen Agent 進行智能創作（請參考 storyboard.py）。
• AI動畫視頻製作：可結合 Wan 圖像到視頻工具進行製作（請參考 assets/example_videos）。

🔧 技術細節

文檔中未提及具體技術細節，暫不提供。

📄 許可證

本項目使用 apache-2.0 許可證。

👏 致謝

此代碼基於 diffusers 和 OminiControl 構建。非常感謝他們的傑出工作！

📖 引用

@article{he2025anystory,
  title={AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generation},
  author={He, Junjie and Tuo, Yuxiang and Chen, Binghui and Zhong, Chongyang and Geng, Yifeng and Bo, Liefeng},
  journal={arXiv preprint arXiv:2501.09503},
  year={2025}
}

📋 模型信息

📢 最新消息

• [2025/05/01] 我們發佈了 AnyStory 的 FLUX.1-dev 版本的代碼和演示。