Marigold-iid-appearance-v1-1 Open-source Model - Free Implementation of Attributes Such as Image Decomposition and Albedo

Marigold Iid Appearance V1 1

Developed by prs-eth

A single-image intrinsic decomposition model fine-tuned based on Stable Diffusion 2, used to decompose images into albedo, roughness, and metallic properties

3D Vision English#Zero-shot Intrinsic Decomposition #Material Property Prediction #Indoor Scene Analysis

Downloads 847

Release Time : 10/21/2024

Model Overview

This model is designed for single-image intrinsic image decomposition (IID), capable of decomposing input images into material properties such as albedo, roughness, and metallicness, suitable for computer vision and real-world scene analysis.

Model Features

Zero-shot Intrinsic Decomposition

Directly applicable to new image intrinsic decomposition tasks without training

Multi-property Decomposition

Simultaneously predicts three material properties: albedo, roughness, and metallicness

High-resolution Processing

Effectively processes images with resolutions up to approximately 768 pixels, suitable for most application scenarios

Uncertainty Estimation

Generates uncertainty maps for each property when using multiple ensemble predictions

Model Capabilities

Image Intrinsic Decomposition

Material Property Analysis

Computer Vision Processing

Real-world Scene Image Analysis

Use Cases

Computer Vision

Indoor Scene Analysis

Analyze material properties of indoor scenes for virtual or augmented reality applications

Obtain albedo, roughness, and metallicness information of the scene

Product Material Analysis

Analyze surface material characteristics of products for industrial design or quality control

Obtain albedo, roughness, and metallicness parameters of product surfaces

Graphics

Material Editing

Perform non-destructive material editing by modifying decomposition results

Independently adjust albedo, roughness, or metallicness properties

🚀 Marigold Intrinsic Image Decomposition (IID) Appearance v1-1 Model Card

This is a model card for the marigold-iid-appearance-v1-1 model, which is used for single-image Intrinsic Image Decomposition (IID). It can generate an estimated intrinsic image decomposition of an input image.

🚀 Quick Start

This model type (appearance) is trained to perform InteriorVerse decomposition into Albedo and two BRDF material properties: roughness and metallicity. Both the input image and the output albedo are in the sRGB color space. For an alternative model type (lighting) that performs decomposition into Albedo, Diffuse shading, and Non-diffuse residual, click here.

Play with the interactive Hugging Face Spaces demo: check out how the model works with example images or upload your own.
Use it with diffusers to compute the results with a few lines of code.
Get to the bottom of things with our official codebase.

✨ Features

This model is fine - tuned from the stable - diffusion - 2 model as described in our papers:
- CVPR'2024 paper titled "Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation"
- Journal extension titled "Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis"

📚 Documentation

Model Details

Property	Details
Developed by	Bingxin Ke, Kevin Qu, Tianfu Wang, Nando Metzger, Shengyu Huang, Bo Li, Anton Obukhov, Konrad Schindler
Model Type	Generative latent diffusion-based intrinsic image decomposition (appearance: albedo, roughness, and metallicity) from a single image
Language	English
License	CreativeML Open RAIL++-M License
Model Description	This model can be used to generate an estimated intrinsic image decomposition of an input image. - Resolution: Even though any resolution can be processed, the model inherits the base diffusion model's effective resolution of roughly 768 pixels. This means that for optimal predictions, any larger input image should be resized to make the longer side 768 pixels before feeding it into the model. - Steps and scheduler: This model was designed for usage with DDIM scheduler and between 1 and 50 denoising steps. - Outputs: - Albedo: The predicted values are between 0 and 1, sRGB space. - Roughness and metallicity: The predicted values are between 0 and 1, linear space. - Uncertainty maps: Produced for each modality only when multiple predictions are ensembled with ensemble size larger than 2.
Resources for more information	Project Website, Paper, Code

Cite as

@misc{ke2025marigold,
  title={Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis},
  author={Bingxin Ke and Kevin Qu and Tianfu Wang and Nando Metzger and Shengyu Huang and Bo Li and Anton Obukhov and Konrad Schindler},
  year={2025},
  eprint={2505.09358},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

@InProceedings{ke2023repurposing,
  title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation},
  author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}

📄 License

This model is licensed under the CreativeML Open RAIL++-M License.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご