DMM Open-source Text-to-Image Generation Model - Integrates Multi-domain Pretrained Models, with Superpowerful Functions!

DMM

Developed by MCG-NJU

DMM is a score distillation-based model fusion paradigm that compresses multiple pre-trained models from different domains into a versatile text-to-image generation model.

Image Generation Open Source License:Apache-2.0 #Multi-style fusion #Text-to-image generation #Model distillation

Downloads 115

Release Time : 4/18/2025

Model Overview

By fusing multiple pre-trained models with different styles (including realistic style, Asian portraits, anime style, illustrations, etc.), this model achieves multi-style image generation capability in a single model.

Model Features

Multi-style fusion

Integrates 8 pre-trained models with different styles, capable of generating images in various styles such as realistic, anime, and illustrations.

Distillation-based model fusion

Uses score distillation to compress multiple models into a single model while preserving the characteristics of each source model.

Style interpolation

Supports interpolation between different styles to achieve style transition effects.

LoRA compatibility

Can be combined with character LoRA to generate multi-style images of specific characters.

Model Capabilities

Text-to-image generation

Multi-style image generation

Style interpolation

Integration with LoRA models

Use Cases

Creative design

Multi-style portrait generation

Generates portrait images in different styles based on text prompts.

Can produce portraits in various styles such as realistic, anime, and illustrations.

Style exploration

Explores transition effects between different artistic styles through style interpolation.

Achieves smooth style transition effects.

Content creation

Multi-style character presentation

Combines with character LoRA to generate the same character in different styles.

Applies different artistic styles while maintaining character features.

🚀 DMM: Building a Versatile Image Generation Model via Distillation-Based Model Merging

This project proposes a score distillation based model merging paradigm DMM, which compresses multiple models into a single versatile text-to-image model.

🚀 Quick Start

The project provides an online demo and usage code examples. You can access the online demo through the following link: Online Demo. For detailed usage, please refer to GitHub Repository.

✨ Features

Model Merging: The DMM paradigm compresses multiple models into a single versatile text-to-image model.
Multi - domain Integration: This checkpoint merges pre - trained models from many different domains, such as realistic style, Asian portrait, anime style, illustration, etc.

📦 Installation

The original README does not provide installation steps, so this section is skipped.

💻 Usage Examples

Basic Usage

import torch
from modeling.dmm_pipeline import StableDiffusionDMMPipeline

pipe = StableDiffusionDMMPipeline.from_pretrained("path/to/pipeline/checkpoint", torch_dtype=torch.float16, use_safetensors=True)
pipe = pipe.to("cuda")

# select model index
model_id = 5
output = pipe(
    prompt="portrait photo of a girl, long golden hair, flowers, best quality",
    negative_prompt="worst quality,low quality,normal quality,lowres,watermark,nsfw",
    width=512,
    height=512,
    num_inference_steps=25,
    guidance_scale=7,
    model_id=model_id,
).images[0]

📚 Documentation

Introduction

We propose a score distillation based model merging paradigm DMM, compressing multiple models into a single versatile T2I model. The method

This checkpoint merges pre-trained models from many different domains, including realistic style, Asian portrait, anime style, illustration, etc. Specifically, the source models are listed below:

Visualization

Results

Results combined with charactor LoRA

Results of interpolation between two styles

🔧 Technical Details

The original README does not provide technical details, so this section is skipped.

📄 License

This project is licensed under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご