Object Detection using Detectron2

author: filip-halt, fzliu

Description

This operator uses Facebook's Detectron2 library to compute bounding boxes, class labels, and class scores for detected objects in a given image.

Code Example

from towhee import pipe, ops, DataCollection

p = (
    pipe.input('path')
        .map('path', 'img', ops.image_decode())
        .map('img', ('boxes', 'classes', 'scores'), ops.object_detection.detectron2(model_name='retinanet_resnet50'))
        .output('img', 'boxes', 'classes', 'scores')
)

DataCollection(p('./example.jpg')).show()

Factory Constructor

Create the operator via the following factory method

object_detection.detectron2(model_name='retinanet_resnet50', thresh=0.5, num_classes=1000, skip_preprocess=False)

Parameters:

model_name: str

A string indicating which model to use. Available options:

1. faster_rcnn_resnet50_c4
2. faster_rcnn_resnet50_dc5
3. faster_rcnn_resnet50_fpn
4. faster_rcnn_resnet101_c4
5. faster_rcnn_resnet101_dc5
6. faster_rcnn_resnet101_fpn
7. faster_rcnn_resnext101
8. retinanet_resnet50
9. retinanet_resnet101

thresh: float

The threshold value for which an object is detected (default value: 0.5). Set this value lower to detect more objects at the expense of accuracy, or higher to reduce the total number of detections but increase the quality of detected objects.

Interface

This operator takes an image as input. It first detects the objects appeared in the image, and generates a bounding box around each object.

Parameters:

​ img: towhee._types.Image Image data wrapped in a (as a Towhee Image).

Return: List[numpy.ndarray[4], ...], List[str], numpy.ndarray

The return value is a tuple of (boxes, classes, scores). boxes is a list of bounding boxes. Each bounding box is represented as a 1-dimensional numpy array consisting of the top-left and the bottom-right corners, i.e. numpy.ndarray([x1, y1, x2, y2]). classes is a list of prediction labels for each bounding box. scores is a list of confidence scores corresponding to each class and bounding box.