Glpn Nyu

🚀 GLPN在NYUv2上微调模型

本项目是基于NYUv2数据集微调的Global-Local Path Networks (GLPN) 模型，用于单目深度估计。该模型由Kim等人在论文 Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth 中提出，并首次在 此仓库 发布。

需要说明的是，发布GLPN的团队并未为此模型编写模型卡片，此卡片由Hugging Face团队编写。

✨ 主要特性

• 架构设计：GLPN以SegFormer为骨干网络，并在其基础上添加了一个轻量级头部用于深度估计。

🚀 快速开始

你可以使用此原始模型进行单目深度估计。你可以在 模型中心 查找针对你感兴趣任务的微调版本。

💻 使用示例

基础用法

from transformers import GLPNImageProcessor, GLPNForDepthEstimation
import torch
import numpy as np
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

processor = GLPNImageProcessor.from_pretrained("vinvino02/glpn-nyu")
model = GLPNForDepthEstimation.from_pretrained("vinvino02/glpn-nyu")

# prepare image for the model
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)
    predicted_depth = outputs.predicted_depth

# interpolate to original size
prediction = torch.nn.functional.interpolate(
    predicted_depth.unsqueeze(1),
    size=image.size[::-1],
    mode="bicubic",
    align_corners=False,
)

# visualize the prediction
output = prediction.squeeze().cpu().numpy()
formatted = (output * 255 / np.max(output)).astype("uint8")
depth = Image.fromarray(formatted)

更多代码示例请参考 文档。

📚 详细文档

BibTeX引用和引用信息

@article{DBLP:journals/corr/abs-2201-07436,
  author    = {Doyeon Kim and
               Woonghyun Ga and
               Pyunghwan Ahn and
               Donggyu Joo and
               Sehwan Chun and
               Junmo Kim},
  title     = {Global-Local Path Networks for Monocular Depth Estimation with Vertical
               CutDepth},
  journal   = {CoRR},
  volume    = {abs/2201.07436},
  year      = {2022},
  url       = {https://arxiv.org/abs/2201.07436},
  eprinttype = {arXiv},
  eprint    = {2201.07436},
  timestamp = {Fri, 21 Jan 2022 13:57:15 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2201-07436.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

📄 许可证

本项目采用Apache-2.0许可证。