Marigold - IID Lighting V1 - 1 Open - source Model: Freely Decompose Images into Multiple Parts such as Albedo

Marigold Iid Lighting V1 1

Developed by prs-eth

A single-image intrinsic image decomposition model fine-tuned based on Stable Diffusion, capable of decomposing images into albedo, diffuse shading, and non-diffuse residual components

3D Vision English#Intrinsic Image Decomposition #Zero-shot Learning #Lighting Analysis

Downloads 520

Release Time : 12/16/2024

Model Overview

This model is designed for single-image intrinsic image decomposition (IID), decomposing input images into three components: albedo, diffuse shading, and non-diffuse residual, suitable for computer vision and image analysis tasks.

Model Features

Intrinsic Image Decomposition

Decomposes images into three components: albedo, diffuse shading, and non-diffuse residual

Zero-shot Learning

Capable of processing various real-world scene images without specific training

High-Resolution Processing

Recommended to adjust the longer side of input images to 768 pixels for optimal results

Multi-Output Mode

Can generate albedo, diffuse shading, non-diffuse residual, and uncertainty maps

Model Capabilities

Image Decomposition

Lighting Analysis

Albedo Estimation

Diffuse Shading Estimation

Non-Diffuse Residual Estimation

Use Cases

Computer Vision

Intrinsic Image Decomposition

Decompose input images into albedo, diffuse shading, and non-diffuse residual components

Obtain three intrinsic components of the image for subsequent analysis and processing

Lighting Analysis

Extract diffuse shading components from images to analyze lighting conditions

Obtain lighting color information of the scene

Image Processing

Material Editing

Modify surface materials of objects by altering the albedo component

Achieve editing and replacement of object surface materials

🚀 Marigold Intrinsic Image Decomposition (IID) Lighting v1-1 Model Card

This model card is for the marigold-iid-lighting-v1-1 model, which is designed for single-image Intrinsic Image Decomposition (IID). It offers a solution for analyzing images by decomposing them into key components, facilitating in - depth image analysis and understanding.

🚀 Quick Start

This is a model card for the marigold-iid-lighting-v1-1 model for single - image Intrinsic Image Decomposition (IID). The model is fine - tuned from the stable - diffusion - 2 [model](https://huggingface.co/stabilityai/stable - diffusion - 2) as described in our papers:

CVPR'2024 paper titled "Repurposing Diffusion - Based Image Generators for Monocular Depth Estimation"
Journal extension titled "Marigold: Affordable Adaptation of Diffusion - Based Image Generators for Image Analysis"

✨ Features

Interactive Demo: Play with the interactive [Hugging Face Spaces demo](https://huggingface.co/spaces/prs - eth/marigold - iid) to see how the model works with example images or upload your own.
Easy Integration: Use it with [diffusers](https://huggingface.co/docs/diffusers/using - diffusers/marigold_usage) to compute the results with a few lines of code.
Official Codebase: Dive deeper into the model with our [official codebase](https://github.com/prs - eth/marigold).

📚 Documentation

Using the model

This model type (lighting) is trained to perform HyperSim decomposition into Albedo, Diffuse shading, and Non - diffuse residual. This decomposition aligns with the intrinsic residual model \(I = A*S+R\), where the image \(I\) is composed of a three - channel albedo \(A\), a three - channel diffuse shading component \(S\) (representing illumination color), and an additive three - channel residual term \(R\) capturing non - diffuse effects. The input is in the sRGB color space, while all outputs are in linear space. For an alternative model type (appearance) that performs decomposition into Albedo, Roughness, and Metallicity, click [here](https://huggingface.co/prs - eth/marigold - iid - appearance - v1 - 1).

Model Details

Property	Details
Developed by	Bingxin Ke, [Kevin Qu](https://ch.linkedin.com/in/kevin - qu - b3417621b), Tianfu Wang, Nando Metzger, Shengyu Huang, Bo Li, Anton Obukhov, Konrad Schindler
Model Type	Generative latent diffusion - based intrinsic image decomposition (lighting: albedo, diffuse shading, and non - diffuse residual) from a single image.
Language	English
License	[CreativeML Open RAIL++ - M License](https://huggingface.co/stabilityai/stable - diffusion - 2/blob/main/LICENSE - MODEL)
Model Description	This model can be used to generate an estimated intrinsic image decomposition of an input image. - Resolution: Even though any resolution can be processed, the model inherits the base diffusion model's effective resolution of roughly 768 pixels. This means that for optimal predictions, any larger input image should be resized to make the longer side 768 pixels before feeding it into the model. - Steps and scheduler: This model was designed for usage with DDIM scheduler and between 1 and 50 denoising steps. - Outputs: - Albedo: The predicted values are between 0 and 1, linear space. - Diffuse shading: The predicted values are between 0 and 1, linear space. - Non - diffuse residual: The predicted values are between 0 and 1, linear space. - Uncertainty maps: Produced for each modality only when multiple predictions are ensembled with ensemble size larger than 2.
Resources for more information	Project Website, Paper, [Code](https://github.com/prs - eth/marigold)
Cite as	bibtex<br>@misc{ke2025marigold,<br> title={Marigold: Affordable Adaptation of Diffusion - Based Image Generators for Image Analysis},<br> author={Bingxin Ke and Kevin Qu and Tianfu Wang and Nando Metzger and Shengyu Huang and Bo Li and Anton Obukhov and Konrad Schindler},<br> year={2025},<br> eprint={2505.09358},<br> archivePrefix={arXiv},<br> primaryClass={cs.CV}<br>}<br><br>@InProceedings{ke2023repurposing,<br> title={Repurposing Diffusion - Based Image Generators for Monocular Depth Estimation},<br> author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},<br> booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},<br> year={2024}<br>}<br>

📄 License

This model is licensed under the [CreativeML Open RAIL++ - M License](https://huggingface.co/stabilityai/stable - diffusion - 2/blob/main/LICENSE - MODEL).

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご