Marigold-depth-v1-0 Open-source Model - Free Monocular Image Depth Prediction in Natural Scenes

Marigold Depth V1 0

Developed by prs-eth

A monocular image depth estimation model fine-tuned based on Stable Diffusion, featuring affine invariance for depth prediction in natural scenes

3D Vision EnglishOpen Source License:Apache-2.0 #Zero-shot depth estimation #Diffusion model fine-tuning #Affine-invariant depth

Downloads 92.50k

Release Time : 12/5/2023

Model Overview

This model generates estimated depth maps from single images, fine-tuned from Stable Diffusion 2, supporting zero-shot learning

Model Features

Affine-invariant depth estimation

The model predicts depth values between 0 and 1 with affine invariance, suitable for scenes at different scales

Zero-shot learning capability

Capable of depth estimation without training data for specific scenes

Efficient inference

Supports 1-step inference for good results, or 10-50 steps for more precise outcomes

Uncertainty estimation

Can generate uncertainty maps when integrating multiple predictions

Model Capabilities

Monocular image depth estimation

Natural scene analysis

Depth map generation

Uncertainty quantification

Use Cases

Computer vision

3D scene reconstruction

Estimating scene depth information from single images

Applicable for 3D modeling and scene understanding

Augmented reality

Providing depth information for AR applications

Enables more realistic virtual-real integration

Robotic vision

Autonomous navigation

Providing environmental depth perception for robots

Assists in path planning and obstacle avoidance

🚀 Marigold Depth v1-0 Model Card

This is a model card for the marigold-depth-v1-0 model, which is used for monocular depth estimation from a single image, offering valuable insights into image analysis and computer vision.

NEW: Marigold Depth v1-1 Model

This is a model card for the marigold-depth-v1-0 model for monocular depth estimation from a single image. The model is fine-tuned from the stable-diffusion-2 model as described in our papers:

CVPR'2024 paper titled "Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation"
Journal extension titled "Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis"

🚀 Quick Start

Using the model

Play with the interactive Hugging Face Spaces demo: check out how the model works with example images or upload your own.
Use it with diffusers to compute the results with a few lines of code.
Get to the bottom of things with our official codebase.

✨ Features

Model Details

Property	Details
Developed by	Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, Konrad Schindler
Model Type	Generative latent diffusion-based affine-invariant monocular depth estimation from a single image
Language	English
License	Apache License License Version 2.0
Model Description	This model can be used to generate an estimated depth map of an input image. - Resolution: Even though any resolution can be processed, the model inherits the base diffusion model's effective resolution of roughly 768 pixels. This means that for optimal predictions, any larger input image should be resized to make the longer side 768 pixels before feeding it into the model. - Steps and scheduler: This model was designed for usage with the DDIM scheduler and between 10 and 50 denoising steps. It is possible to obtain good predictions with just one step by overriding the `"timestep_spacing": "trailing"` setting in the scheduler configuration file or by adding `pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")` after the pipeline is loaded in the code before the first usage. For compatibility reasons we kept this `v1-0` model identical to the paper setting and provided a newer v1-1 model with optimal settings for all possible step configurations. - Outputs: - Affine-invariant depth map: The predicted values are between 0 and 1, interpolating between the near and far planes of the model's choice. - Uncertainty map: Produced only when multiple predictions are ensembled with ensemble size larger than 2.
Resources for more information	Project Website, Paper, Code

📄 License

The model is released under the Apache License License Version 2.0.

@InProceedings{ke2023repurposing,
  title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation},
  author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}

@misc{ke2025marigold,
  title={Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis},
  author={Bingxin Ke and Kevin Qu and Tianfu Wang and Nando Metzger and Shengyu Huang and Bo Li and Anton Obukhov and Konrad Schindler},
  year={2025},
  eprint={2505.09358},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご