Segmoe SD 4x2 V0

🚀 SegMoE-SD-4x2-v0：Segmind擴散專家混合模型

SegMoE-SD-4x2-v0是一個基於Segmind的擴散專家混合模型，它利用segmoe框架，從4個SD1.5專家模型生成，無需訓練。SegMoE是一個強大的框架，可在幾分鐘內將多個Stable Diffusion模型動態組合成一個專家混合模型，無需訓練。該框架允許即時創建更大的模型，這些模型具有更廣泛的知識、更好的一致性和更高的圖像質量。

🚀 快速開始

本模型可以通過 segmoe 庫使用。

安裝segmoe

請確保通過以下命令安裝 segmoe：

pip install segmoe

使用示例代碼

from segmoe import SegMoEPipeline

pipeline = SegMoEPipeline("segmind/SegMoE-SD-4x2-v0", device = "cuda")

prompt = "cosmic canvas, orange city background, painting of a chubby cat"
negative_prompt = "nsfw, bad quality, worse quality"
img = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=1024,
    width=1024,
    num_inference_steps=25,
    guidance_scale=7.5,
).images[0]
img.save("image.png")

✨ 主要特性

• 融合多專家知識：受益於多個微調專家模型的知識。
• 無需訓練：無需額外的訓練過程。
• 更好的數據適應性：對數據有更好的適應性。
• 模型可升級：可以通過使用更好的微調模型作為專家之一來升級模型。

📦 安裝指南

確保通過以下命令安裝 segmoe 庫：

pip install segmoe

💻 使用示例

基礎用法

from segmoe import SegMoEPipeline

pipeline = SegMoEPipeline("segmind/SegMoE-SD-4x2-v0", device = "cuda")

prompt = "cosmic canvas, orange city background, painting of a chubby cat"
negative_prompt = "nsfw, bad quality, worse quality"
img = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=1024,
    width=1024,
    num_inference_steps=25,
    guidance_scale=7.5,
).images[0]
img.save("image.png")

📚 詳細文檔

配置信息

用於創建此模型的配置如下：

base_model: SG161222/Realistic_Vision_V6.0_B1_noVAE
num_experts: 4
moe_layers: all
num_experts_per_tok: 2
experts:
  - source_model: SG161222/Realistic_Vision_V6.0_B1_noVAE
    positive_prompt: "cinematic, portrait, photograph, instagram, fashion, movie, macro shot, 8K, RAW, hyperrealistic, ultra realistic,"
    negative_prompt: " (deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime), text, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck"
  - source_model: dreamlike-art/dreamlike-anime-1.0
    positive_prompt: "photo anime, masterpiece, high quality, absurdres, 1girl, 1boy, waifu, chibi"
    negative_prompt: "simple background, duplicate, retro style, low quality, lowest quality, 1980s, 1990s, 2000s, 2005 2006 2007 2008 2009 2010 2011 2012 2013, bad anatomy, bad proportions, extra digits, lowres, username, artist name, error, duplicate, watermark, signature, text, extra digit, fewer digits, worst quality, jpeg artifacts, blurry"
  - source_model: Lykon/dreamshaper-8
    positive_prompt: "bokeh, intricate, elegant, sharp focus, soft lighting, vibrant colors, dreamlike, fantasy, artstation, concept art"
    negative_prompt: "low quality, lowres, jpeg artifacts, signature, bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame, tiling, bad art, deformed, mutated, blurry, fuzzy, misshaped, mutant, gross, disgusting, ugly, watermark, watermarks"
  - source_model: dreamlike-art/dreamlike-diffusion-1.0
    positive_prompt: "dreamlikeart, a grungy woman with rainbow hair, travelling between dimensions, dynamic pose, happy, soft eyes and narrow chin, extreme bokeh, dainty figure, long hair straight down, torn kawaii shirt and baggy jeans, In style of by Jordan Grimmer and greg rutkowski, crisp lines and color, complex background, particles, lines, wind, concept art, sharp focus, vivid colors"
    negative_prompt: "nude, naked, low quality, lowres, jpeg artifacts, signature, bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame"

其他變體

我們在Hugging Face上發佈了3個合併模型：

• SegMoE 4x2：包含四個Stable Diffusion XL專家模型。
• SegMoE 2x1：包含兩個Stable Diffusion XL專家模型。

模型描述

適用範圍外的使用

SegMoE-SD-4x2-v0模型不適用於創建人物、事件或現實世界信息的事實性或準確表示。它不適合需要高精度和準確性的任務。

🔧 技術細節

優點

• 受益於多個微調專家模型的知識。
• 無需訓練。
• 對數據有更好的適應性。
• 可以通過使用更好的微調模型作為專家之一來升級模型。

侷限性

• 儘管該模型在圖像保真度和一致性方面有所改進，但在未訓練的情況下，它並不比任何一個專家模型有顯著的提升，並且依賴於專家模型的知識。
• 該模型尚未針對速度進行優化。
• 該框架尚未針對內存使用進行優化。

📄 許可證

本模型使用 Apache 2.0 許可證。

📖 引用

@misc{segmoe,
  author = {Yatharth Gupta, Vishnu V Jaddipal, Harish Prabhala},
  title = {SegMoE},
  year = {2024},
  publisher = {HuggingFace},
  journal = {HuggingFace Models},
  howpublished = {\url{https://huggingface.co/segmind/SegMoE-SD-4x2-v0}}
}