Tensorflow DrawBoundingBoxes C++

Question

I am detecting objects using Tensorflow in C++. It does work well and i want to draw the boxes to have some visual feedback.
There is the operation tensorflow::ops::DrawBoundingBoxes that would let me do this but the problem is :

• I don't understand what the input values should be for the boxes. What does this mean :

  boxes: 3-D with shape [batch, num_bounding_boxes, 4] containing bounding boxes.
  

• I could not find an example that uses this operation in C++ anywhere, like almost as if this ops does not exist.

Is there an example somewhere in C++ where this ops is used? It sounds a basic things to do for a tutorial or to debug.

Answer 1

In case if you are still on this question, I have written my own implementation of this operation using OpenCV basic methods. It also supports captioning the boxes with corresponding class labels.

/** Draw bounding box and add caption to the image.
 *  Boolean flag _scaled_ shows if the passed coordinates are in relative units (true by default in tensorflow detection)
 */
void drawBoundingBoxOnImage(Mat &image, double yMin, double xMin, double yMax, double xMax, double score, string label, bool scaled=true) {
    cv::Point tl, br;
    if (scaled) {
        tl = cv::Point((int) (xMin * image.cols), (int) (yMin * image.rows));
        br = cv::Point((int) (xMax * image.cols), (int) (yMax * image.rows));
    } else {
        tl = cv::Point((int) xMin, (int) yMin);
        br = cv::Point((int) xMax, (int) yMax);
    }
    cv::rectangle(image, tl, br, cv::Scalar(0, 255, 255), 1);

    // Ceiling the score down to 3 decimals (weird!)
    float scoreRounded = floorf(score * 1000) / 1000;
    string scoreString = to_string(scoreRounded).substr(0, 5);
    string caption = label + " (" + scoreString + ")";

    // Adding caption of type "LABEL (X.XXX)" to the top-left corner of the bounding box
    int fontCoeff = 12;
    cv::Point brRect = cv::Point(tl.x + caption.length() * fontCoeff / 1.6, tl.y + fontCoeff);
    cv::rectangle(image, tl, brRect, cv::Scalar(0, 255, 255), -1);
    cv::Point textCorner = cv::Point(tl.x, tl.y + fontCoeff * 0.9);
    cv::putText(image, caption, textCorner, FONT_HERSHEY_SIMPLEX, 0.4, cv::Scalar(255, 0, 0));
}

/** Draw bounding boxes and add captions to the image.
 *  Box is drawn only if corresponding score is higher than the _threshold_.
 */
void drawBoundingBoxesOnImage(Mat &image,
                              tensorflow::TTypes<float>::Flat scores,
                              tensorflow::TTypes<float>::Flat classes,
                              tensorflow::TTypes<float,3>::Tensor boxes,
                              map<int, string> labelsMap, double threshold=0.5) {
    for (int j = 0; j < scores.size(); j++)
        if (scores(j) > threshold)
            drawBoundingBoxOnImage(image, boxes(0,j,0), boxes(0,j,1), boxes(0,j,2), boxes(0,j,3), scores(j), labelsMap[classes(j)]);
}

The complete example is here.