Coreml - Depth Anything Small Open-Source Depth Estimation Model - Achieve Advanced Depth Estimation with Multi-Image Training

Coreml Depth Anything Small

Developed by apple

Depth Anything is a depth estimation model based on the DPT architecture and DINOv2 backbone network, trained on approximately 62 million images, achieving state-of-the-art results in relative and absolute depth estimation tasks.

3D Vision Open Source License:Apache-2.0 #Monocular depth estimation #Large-scale unsupervised training #Mobile optimization

Downloads 51

Release Time : 5/3/2024

Model Overview

This model adopts the DPT architecture and is based on the DINOv2 backbone network, focusing on image depth estimation tasks, capable of predicting depth information from a single image.

Model Features

Large-scale training data

Trained on approximately 62 million images, ensuring strong generalization capabilities.

Efficient architecture

Based on DPT architecture and DINOv2 backbone, achieving efficient inference while maintaining accuracy.

Multi-platform support

Provides Core ML format models for efficient operation on Apple devices.

Precision optimization

Offers both Float32 and Float16 precision versions to balance accuracy and performance needs.

Model Capabilities

Monocular depth estimation

Relative depth prediction

Absolute depth prediction

Image depth analysis

Use Cases

Computer vision

Augmented reality

Provides scene depth information for AR applications.

Enhances realism in AR object placement and interaction.

3D reconstruction

Generates depth maps from single images.

Assists in 3D scene reconstruction.

Autonomous driving

Assists vehicles in perceiving surrounding environments.

Provides scene depth information.

🚀 Depth Anything Core ML Models

Depth Anything is a state - of - the - art model for depth estimation, leveraging large - scale unlabeled data to achieve excellent results.

🚀 Quick Start

Download

First, install huggingface-cli:

brew install huggingface-cli

To download one of the .mlpackage folders to the models directory:

huggingface-cli download \
  --local-dir models --local-dir-use-symlinks False \
  apple/coreml-depth-anything-small \
  --include "DepthAnythingSmallF16.mlpackage/*"

To download everything, skip the --include argument.

Integrate in Swift apps

The huggingface/coreml-examples repository contains sample Swift code for coreml-depth-anything-small and other models. See the instructions there to build the demo app, which shows how to use the model in your own Swift apps.

✨ Features

Depth Anything model was introduced in the paper Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data by Lihe Yang et al. and first released in this repository.

📚 Documentation

Model description

Depth Anything leverages the DPT architecture with a DINOv2 backbone.

The model is trained on ~62 million images, obtaining state - of - the - art results for both relative and absolute depth estimation.

drawing

Depth Anything overview. Taken from the original paper.

Evaluation - Variants

Property	Details
Model Type	Depth Anything with Core ML variants
Training Data	~62 million images

Variant	Parameters	Size (MB)	Weight precision	Act. precision	abs-rel error	abs-rel reference
small-original (PyTorch)	24.8M	99.2	Float32	Float32
DepthAnythingSmallF32	24.8M	99.0	Float32	Float32	0.0073	small-original
DepthAnythingSmallF16	24.8M	45.8	Float16	Float16	0.0077	small-original

Evaluation - Inference time

The following results use the small-float16 variant.

Device	OS	Inference time (ms)	Dominant compute unit
iPhone 12 Pro Max	18.0	31.10	Neural Engine
iPhone 15 Pro Max	17.4	33.90	Neural Engine
MacBook Pro (M1 Max)	15.0	32.80	Neural Engine
MacBook Pro (M3 Max)	15.0	24.58	Neural Engine

📄 License

This project is licensed under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご