Detr Resnet 50 Dc5

🚀 DETR (End-to-End Object Detection) model with ResNet-50 backbone (dilated C5 stage)

The DETR model is an end-to-end object detection model trained on COCO 2017, which uses a transformer architecture to simplify the object detection process.

🚀 Quick Start

The DETR (DEtection TRansformer) model was trained end-to-end on COCO 2017 object detection (118k annotated images). It was introduced in the paper End-to-End Object Detection with Transformers by Carion et al. and first released in this repository.

Disclaimer: The team releasing DETR did not write a model card for this model so this model card has been written by the Hugging Face team.

✨ Features

The DETR model is an encoder-decoder transformer with a convolutional backbone. Two heads are added on top of the decoder outputs for object detection: a linear layer for class labels and a MLP (multi-layer perceptron) for bounding boxes. It uses object queries to detect objects in an image, with 100 object queries for COCO.

The model is trained using a "bipartite matching loss". It compares the predicted classes and bounding boxes of each of the N = 100 object queries to the ground truth annotations, padded to the same length N. The Hungarian matching algorithm creates an optimal one-to-one mapping between queries and annotations. Standard cross-entropy (for classes) and a linear combination of the L1 and generalized IoU loss (for bounding boxes) are used to optimize the model parameters.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

from transformers import DetrFeatureExtractor, DetrForObjectDetection
from PIL import Image
import requests

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

feature_extractor = DetrFeatureExtractor.from_pretrained('facebook/detr-resnet-50-dc5')
model = DetrForObjectDetection.from_pretrained('facebook/detr-resnet-50-dc5')

inputs = feature_extractor(images=image, return_tensors="pt")
outputs = model(**inputs)

# model predicts bounding boxes and corresponding COCO classes
logits = outputs.logits
bboxes = outputs.pred_boxes

Currently, both the feature extractor and model support PyTorch.

📚 Documentation

You can use the raw model for object detection. See the model hub to look for all available DETR models.

🔧 Technical Details

Training Data

The DETR model was trained on COCO 2017 object detection, a dataset with 118k/5k annotated images for training/validation respectively.

Training Procedure

Preprocessing

The exact details of preprocessing of images during training/validation can be found here.

Images are resized/rescaled so that the shortest side is at least 800 pixels and the largest side at most 1333 pixels, and normalized across the RGB channels with the ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225).

Training

The model was trained for 300 epochs on 16 V100 GPUs. This takes 3 days, with 4 images per GPU (a total batch size of 64).

Evaluation Results

This model achieves an AP (average precision) of 43.3 on COCO 2017 validation. For more details regarding evaluation results, refer to table 1 of the original paper.

📄 License

The model is licensed under the Apache-2.0 license.

BibTeX entry and citation info

@article{DBLP:journals/corr/abs-2005-12872,
  author    = {Nicolas Carion and
               Francisco Massa and
               Gabriel Synnaeve and
               Nicolas Usunier and
               Alexander Kirillov and
               Sergey Zagoruyko},
  title     = {End-to-End Object Detection with Transformers},
  journal   = {CoRR},
  volume    = {abs/2005.12872},
  year      = {2020},
  url       = {https://arxiv.org/abs/2005.12872},
  archivePrefix = {arXiv},
  eprint    = {2005.12872},
  timestamp = {Thu, 28 May 2020 17:38:09 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2005-12872.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}