Zoedepth-Kitti Open-source Vision Model - Monocular Depth Estimation, Supporting Zero-shot Transfer Metrics

Zoedepth Kitti

Developed by Intel

ZoeDepth is a vision model for monocular depth estimation, fine-tuned on the KITTI dataset, capable of achieving zero-shot transfer for metric depth estimation.

3D Vision

Transformers

Open Source License:MIT #Monocular Depth Estimation #Zero-shot Transfer #Metric Depth

Downloads 7,037

Release Time : 4/30/2024

Model Overview

ZoeDepth extends the DPT framework for metric (absolute) depth estimation, achieving state-of-the-art results after fine-tuning on the KITTI dataset. The model can estimate image depth with actual metric values.

Model Features

Zero-shot Transfer Capability

Achieves zero-shot transfer by combining relative and metric depth, applicable without training for specific scenarios.

Metric Depth Estimation

Capable of outputting actual metric depth values, not just relative depth relationships.

State-of-the-art Performance

Achieves state-of-the-art depth estimation results after fine-tuning on the KITTI dataset.

Model Capabilities

Monocular Depth Estimation

Zero-shot Transfer

Metric Depth Prediction

Use Cases

Autonomous Driving

Road Scene Depth Estimation

Used for depth perception of road environments in autonomous driving systems

Performs excellently on the KITTI dataset

Robotic Vision

Environmental 3D Perception

Provides robots with depth information of their surroundings

🚀 ZoeDepth (fine-tuned on KITT)

ZoeDepth model fine-tuned on the KITTI dataset, achieving state-of-the-art results in metric depth estimation.

🚀 Quick Start

ZoeDepth extends the DPT framework for metric (also called absolute) depth estimation. It was introduced in the paper ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth by Shariq et al. and first released in this repository.

Disclaimer: The team releasing ZoeDepth did not write a model card for this model so this model card has been written by the Hugging Face team.

✨ Features

Adapts DPT for metric depth estimation.
Able to estimate depth in actual metric values.
Obtains state-of-the-art results in depth estimation.

📚 Documentation

Model description

ZoeDepth adapts DPT, a model for relative depth estimation, for so-called metric (also called absolute) depth estimation. This means that the model is able to estimate depth in actual metric values.

drawing

ZoeDepth architecture. Taken from the original paper.

Intended uses & limitations

You can use the raw model for tasks like zero-shot monocular depth estimation. See the model hub to look for other versions on a task that interests you.

💻 Usage Examples

Basic Usage

The easiest is to leverage the pipeline API which abstracts away the complexity for the user:

from transformers import pipeline
from PIL import Image
import requests

# load pipe
depth_estimator = pipeline(task="depth-estimation", model="Intel/zoedepth-kitti")

# load image
url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

# inference
outputs = depth_estimator(image)
depth = outputs.depth

For more code examples, we refer to the documentation.

BibTeX entry and citation info

@misc{bhat2023zoedepth,
      title={ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth}, 
      author={Shariq Farooq Bhat and Reiner Birkl and Diana Wofk and Peter Wonka and Matthias Müller},
      year={2023},
      eprint={2302.12288},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

📄 License

This model is released under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご