Segmoe SD 4x2 V0

🚀 SegMoE-SD-4x2-v0：Segmind扩散专家混合模型

SegMoE-SD-4x2-v0是一个基于Segmind的扩散专家混合模型，它利用segmoe框架，从4个SD1.5专家模型生成，无需训练。SegMoE是一个强大的框架，可在几分钟内将多个Stable Diffusion模型动态组合成一个专家混合模型，无需训练。该框架允许即时创建更大的模型，这些模型具有更广泛的知识、更好的一致性和更高的图像质量。

🚀 快速开始

本模型可以通过 segmoe 库使用。

安装segmoe

请确保通过以下命令安装 segmoe：

pip install segmoe

使用示例代码

from segmoe import SegMoEPipeline

pipeline = SegMoEPipeline("segmind/SegMoE-SD-4x2-v0", device = "cuda")

prompt = "cosmic canvas, orange city background, painting of a chubby cat"
negative_prompt = "nsfw, bad quality, worse quality"
img = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=1024,
    width=1024,
    num_inference_steps=25,
    guidance_scale=7.5,
).images[0]
img.save("image.png")

✨ 主要特性

• 融合多专家知识：受益于多个微调专家模型的知识。
• 无需训练：无需额外的训练过程。
• 更好的数据适应性：对数据有更好的适应性。
• 模型可升级：可以通过使用更好的微调模型作为专家之一来升级模型。

📦 安装指南

确保通过以下命令安装 segmoe 库：

pip install segmoe

💻 使用示例

基础用法

from segmoe import SegMoEPipeline

pipeline = SegMoEPipeline("segmind/SegMoE-SD-4x2-v0", device = "cuda")

prompt = "cosmic canvas, orange city background, painting of a chubby cat"
negative_prompt = "nsfw, bad quality, worse quality"
img = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=1024,
    width=1024,
    num_inference_steps=25,
    guidance_scale=7.5,
).images[0]
img.save("image.png")

📚 详细文档

配置信息

用于创建此模型的配置如下：

base_model: SG161222/Realistic_Vision_V6.0_B1_noVAE
num_experts: 4
moe_layers: all
num_experts_per_tok: 2
experts:
  - source_model: SG161222/Realistic_Vision_V6.0_B1_noVAE
    positive_prompt: "cinematic, portrait, photograph, instagram, fashion, movie, macro shot, 8K, RAW, hyperrealistic, ultra realistic,"
    negative_prompt: " (deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime), text, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck"
  - source_model: dreamlike-art/dreamlike-anime-1.0
    positive_prompt: "photo anime, masterpiece, high quality, absurdres, 1girl, 1boy, waifu, chibi"
    negative_prompt: "simple background, duplicate, retro style, low quality, lowest quality, 1980s, 1990s, 2000s, 2005 2006 2007 2008 2009 2010 2011 2012 2013, bad anatomy, bad proportions, extra digits, lowres, username, artist name, error, duplicate, watermark, signature, text, extra digit, fewer digits, worst quality, jpeg artifacts, blurry"
  - source_model: Lykon/dreamshaper-8
    positive_prompt: "bokeh, intricate, elegant, sharp focus, soft lighting, vibrant colors, dreamlike, fantasy, artstation, concept art"
    negative_prompt: "low quality, lowres, jpeg artifacts, signature, bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame, tiling, bad art, deformed, mutated, blurry, fuzzy, misshaped, mutant, gross, disgusting, ugly, watermark, watermarks"
  - source_model: dreamlike-art/dreamlike-diffusion-1.0
    positive_prompt: "dreamlikeart, a grungy woman with rainbow hair, travelling between dimensions, dynamic pose, happy, soft eyes and narrow chin, extreme bokeh, dainty figure, long hair straight down, torn kawaii shirt and baggy jeans, In style of by Jordan Grimmer and greg rutkowski, crisp lines and color, complex background, particles, lines, wind, concept art, sharp focus, vivid colors"
    negative_prompt: "nude, naked, low quality, lowres, jpeg artifacts, signature, bad anatomy, extra legs, extra arms, extra fingers, poorly drawn hands, poorly drawn feet, disfigured, out of frame"

其他变体

我们在Hugging Face上发布了3个合并模型：

• SegMoE 4x2：包含四个Stable Diffusion XL专家模型。
• SegMoE 2x1：包含两个Stable Diffusion XL专家模型。

模型描述

适用范围外的使用

SegMoE-SD-4x2-v0模型不适用于创建人物、事件或现实世界信息的事实性或准确表示。它不适合需要高精度和准确性的任务。

🔧 技术细节

优点

• 受益于多个微调专家模型的知识。
• 无需训练。
• 对数据有更好的适应性。
• 可以通过使用更好的微调模型作为专家之一来升级模型。

局限性

• 尽管该模型在图像保真度和一致性方面有所改进，但在未训练的情况下，它并不比任何一个专家模型有显著的提升，并且依赖于专家模型的知识。
• 该模型尚未针对速度进行优化。
• 该框架尚未针对内存使用进行优化。

📄 许可证

本模型使用 Apache 2.0 许可证。

📖 引用

@misc{segmoe,
  author = {Yatharth Gupta, Vishnu V Jaddipal, Harish Prabhala},
  title = {SegMoE},
  year = {2024},
  publisher = {HuggingFace},
  journal = {HuggingFace Models},
  howpublished = {\url{https://huggingface.co/segmind/SegMoE-SD-4x2-v0}}
}