marigold-normals-v0-1 Open-source Model - Accurately Predict Surface Normal Maps from a Single RGB Image

Marigold Normals V0 1

Developed by prs-eth

A monocular image normal estimation model fine-tuned based on Stable Diffusion, capable of predicting surface normal maps from a single RGB image

3D Vision EnglishOpen Source License:Apache-2.0 #Monocular Normal Estimation #Zero-shot Learning #Diffusion Model Fine-tuning

Downloads 8,845

Release Time : 4/18/2024

Model Overview

This model is fine-tuned from Stable Diffusion 2 for monocular normal estimation from a single image, outputting surface normal maps and uncertainty maps.

Model Features

Zero-shot Learning

Capable of handling various real-world scene images without specific scene training

High-resolution Processing

Effectively processes images with resolutions around 768 pixels, suitable for diverse application scenarios

Uncertainty Estimation

Generates uncertainty maps to help assess prediction reliability

Easy Integration

Integrated into the diffusers library, accessible via simple API calls

Model Capabilities

Monocular Normal Estimation

Image Analysis

Computer Vision Processing

Real-world Scene Adaptation

Use Cases

Computer Vision

3D Scene Reconstruction

Estimates surface normals from single images to assist in 3D scene reconstruction

Generates surface normal information usable for 3D modeling

Augmented Reality

Provides scene geometry information for AR applications

Improves the integration of virtual objects into real scenes

Robotic Vision

Robot Navigation

Provides environmental geometry information for robots

Helps robots understand scene structure and obstacles

🚀 Marigold Normals v0-1 Model Card

This is a model card for the marigold-normals-v0-1 model, which is used for monocular normals estimation from a single image.

🚀 Quick Start

This model is deprecated. It is recommended to use the new Marigold Normals v1-1 Model instead.

Using the model

Interactive Demo: Play with the Hugging Face Spaces demo. You can check out how the model works with example images or upload your own.
Code Integration: Use it with diffusers to compute the results with just a few lines of code.
Explore the Codebase: Get to the bottom of things with our official codebase.

✨ Features

This model is fine - tuned from the stable - diffusion - 2 model for monocular normals estimation from a single image.
It is described in the following papers:
- CVPR'2024 paper titled "Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation".
- Journal extension titled "Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis".

📚 Documentation

Model Details

Property	Details
Developed by	Bingxin Ke, Kevin Qu, Tianfu Wang, Nando Metzger, Shengyu Huang, Bo Li, Anton Obukhov, Konrad Schindler
Model Type	Generative latent diffusion-based normals estimation from a single image
Language	English
License	Apache License License Version 2.0
Model Description	This model can be used to generate an estimated surface normals map of an input image. - Resolution: Even though any resolution can be processed, the model inherits the base diffusion model's effective resolution of roughly 768 pixels. This means that for optimal predictions, any larger input image should be resized to make the longer side 768 pixels before feeding it into the model. - Steps and scheduler: This model was designed for usage with DDIM scheduler and between 10 and 50 denoising steps. - Outputs: - Surface normals map: The predicted values are 3 - dimensional unit vectors in the screen space camera. - Uncertainty map: Produced only when multiple predictions are ensembled with ensemble size larger than 2.
Resources for more information	Project Website, Paper, Code

Cite as

@misc{ke2025marigold,
  title={Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis},
  author={Bingxin Ke and Kevin Qu and Tianfu Wang and Nando Metzger and Shengyu Huang and Bo Li and Anton Obukhov and Konrad Schindler},
  year={2025},
  eprint={2505.09358},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

@InProceedings{ke2023repurposing,
  title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation},
  author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}

📄 License

This model is released under the Apache License License Version 2.0.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご