Marigold-normals-lcm-v0-1 Open Source Model - Easily Predict Surface Normal Maps from a Single Image

Marigold Normals Lcm V0 1

Developed by prs-eth

A monocular normal estimation model fine-tuned using latent consistency distillation for predicting surface normal maps from single images

3D Vision EnglishOpen Source License:Apache-2.0 #Monocular Normal Estimation #Zero-shot Learning #Fast Inference

Downloads 632

Release Time : 4/26/2024

Model Overview

This model generates estimated surface normal maps from single images, based on a diffusion model architecture, suitable for natural scene image analysis

Model Features

Fast Inference

Optimized with latent consistency distillation, capable of completing inference in 1-4 steps

Zero-shot Learning

Processes various natural images without requiring scene-specific training

Uncertainty Estimation

Generates uncertainty maps for normal predictions (requires integrating multiple predictions)

Model Capabilities

Monocular Normal Estimation

Image Surface Analysis

Computer Vision Processing

Use Cases

Computer Vision

3D Scene Reconstruction

Estimates surface normals from single images to assist 3D scene reconstruction

Generated normal maps can be used for subsequent 3D modeling

Augmented Reality

Provides surface geometry information for AR applications

Improves lighting and shadow effects of virtual objects in real scenes

🚀 Marigold Normals LCM v0-1 Model Card

This is a model card for the marigold-normals-lcm-v0-1 model, which is used for monocular normals estimation from a single image. It offers efficient and accurate surface normals prediction.

🚀 Quick Start

This is a model card for the marigold-normals-lcm-v0-1 model for monocular normals estimation from a single image. The model is fine-tuned from the marigold-normals-v0-1 model using the latent consistency distillation method, as described in our papers:

CVPR'2024 paper titled "Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation"
Journal extension titled "Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis"

Using the model

Play with the interactive Hugging Face Spaces demo: check out how the model works with example images or upload your own.
Use it with diffusers to compute the results with a few lines of code.
Get to the bottom of things with our official codebase.

📚 Documentation

Model Details

Property	Details
Developed by	Bingxin Ke, Kevin Qu, Tianfu Wang, Nando Metzger, Shengyu Huang, Bo Li, Anton Obukhov, Konrad Schindler
Model Type	Generative latent diffusion-based normals estimation from a single image
Language	English
License	Apache License License Version 2.0
Model Description	This model can be used to generate an estimated surface normals map of an input image. - Resolution: Even though any resolution can be processed, the model inherits the base diffusion model's effective resolution of roughly 768 pixels. This means that for optimal predictions, any larger input image should be resized to make the longer side 768 pixels before feeding it into the model. - Steps and scheduler: This model was designed for usage with the LCM scheduler and between 1 and 4 denoising steps. - Outputs: - Surface normals map: The predicted values are 3 - dimensional unit vectors in the screen space camera. - Uncertainty map: Produced only when multiple predictions are ensembled with ensemble size larger than 2.
Resources for more information	Project Website, Paper, Code
Cite as	bibtex<br>@misc{ke2025marigold,<br> title={Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis},<br> author={Bingxin Ke and Kevin Qu and Tianfu Wang and Nando Metzger and Shengyu Huang and Bo Li and Anton Obukhov and Konrad Schindler},<br> year={2025},<br> eprint={2505.09358},<br> archivePrefix={arXiv},<br> primaryClass={cs.CV}<br>}<br><br>@InProceedings{ke2023repurposing,<br> title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation},<br> author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},<br> booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},<br> year={2024}<br>}<br>

⚠️ Important Note

This model is deprecated. Use the new Marigold Normals v1 - 1 Model instead. NEW: Marigold Normals v1-1 Model

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご