Search code examples
trackingobject-detection-apifaster-rcnn

Faster R-CNN object detection and deep-sort tracking algorithm integration

I have been trying to integrate the Faster R-CNN object detection model with a deep-sort tracking algorithm. However, for some reason, the tracking algorithm does not perform well which means tracking ID just keeps increasing for the same person.

I have used this repository for building my own script. (check demo.py) deep-sort yolov3

What I did:

  1. 1 detection every 30 frames

  2. created a list for detection scores

  3. created a list for detection bounding boxes (considering the input format of deep-sort)

  4. calling the tracker !!!

                 # tracking and draw bounding boxes
         for i in range(0, len(refine_person_detection)):

             confidence_worker.append(refine_person_detection[i][4]) # scores
             bboxes.append([refine_person_detection[i][0], refine_person_detection[i][2],
                            (refine_person_detection[i][1] - refine_person_detection[i][0]),
                            (refine_person_detection[i][3] - refine_person_detection[i][2])]) # bounding boxes

             features = encoder(frame, bboxes)

             detections = [Detection(bbox, confidence, feature) for bbox, confidence, feature in
                           zip(bboxes, confidence_worker, features)]

             boxes = np.array([d.tlwh for d in detections])
             scores = np.array([d.confidence for d in detections])
             indices = preprocessing.non_max_suppression(boxes, nms_max_overlap, scores)
             detections = [detections[i] for i in indices]

             tracker.predict() # calling the tracker
             tracker.update(detections)

             for track in tracker.tracks:
                 k.append(track)
                 if not track.is_confirmed() or track.time_since_update > 1:
                     continue
                 bbox = track.to_tlbr()
                 cv2.rectangle(frame, (int(bbox[0]), int(bbox[1])), (int(bbox[2]), int(bbox[3])),
                               (255, 255, 255), 2)
                 cv2.putText(frame, str(track.track_id), (int(bbox[0]), int(bbox[1])), 0, 5e-3 * 200,
                             (0, 255, 0), 2)

Here is an example of bad results that tracking ID increases. enter image description here

Thanks in advance for any suggestion

Solution

  • The provided code it correct. However, the detection must be done every frame. Since the deep-sort uses the features within the bounding box for tracking, having a gap between the detection frames caused the issue of increasing numbers for the same person

    P.S: @Mustafa please check the code above with every frame detection, should work. feel free to comment if it did not