Glpn Nyu

🚀 GLPN在NYUv2上微調模型

本項目是基於NYUv2數據集微調的Global-Local Path Networks (GLPN) 模型，用於單目深度估計。該模型由Kim等人在論文 Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth 中提出，並首次在 此倉庫 發佈。

需要說明的是，發佈GLPN的團隊並未為此模型編寫模型卡片，此卡片由Hugging Face團隊編寫。

✨ 主要特性

• 架構設計：GLPN以SegFormer為骨幹網絡，並在其基礎上添加了一個輕量級頭部用於深度估計。

🚀 快速開始

你可以使用此原始模型進行單目深度估計。你可以在 模型中心 查找針對你感興趣任務的微調版本。

💻 使用示例

基礎用法

from transformers import GLPNImageProcessor, GLPNForDepthEstimation
import torch
import numpy as np
from PIL import Image
import requests

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

processor = GLPNImageProcessor.from_pretrained("vinvino02/glpn-nyu")
model = GLPNForDepthEstimation.from_pretrained("vinvino02/glpn-nyu")

# prepare image for the model
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)
    predicted_depth = outputs.predicted_depth

# interpolate to original size
prediction = torch.nn.functional.interpolate(
    predicted_depth.unsqueeze(1),
    size=image.size[::-1],
    mode="bicubic",
    align_corners=False,
)

# visualize the prediction
output = prediction.squeeze().cpu().numpy()
formatted = (output * 255 / np.max(output)).astype("uint8")
depth = Image.fromarray(formatted)

更多代碼示例請參考 文檔。

📚 詳細文檔

BibTeX引用和引用信息

@article{DBLP:journals/corr/abs-2201-07436,
  author    = {Doyeon Kim and
               Woonghyun Ga and
               Pyunghwan Ahn and
               Donggyu Joo and
               Sehwan Chun and
               Junmo Kim},
  title     = {Global-Local Path Networks for Monocular Depth Estimation with Vertical
               CutDepth},
  journal   = {CoRR},
  volume    = {abs/2201.07436},
  year      = {2022},
  url       = {https://arxiv.org/abs/2201.07436},
  eprinttype = {arXiv},
  eprint    = {2201.07436},
  timestamp = {Fri, 21 Jan 2022 13:57:15 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2201-07436.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

📄 許可證

本項目採用Apache-2.0許可證。