OpenCV/Android: Rectangles from Mat I did not use appears on screen

Question

I am trying to make my own face detecting android app, which is based on mixed-processing and face detection tutorial. It runs super slow on my device cca 2.5 FPS on 240x160 resolution.

I was trying to find the problem and make it faster, so one of the things I made was displaying result in greyscale instead of RGB. When I changed it back, I forget to delete line where I was drawing rectangles to Greyscale Mat (mGr).

But for my surprise, those were still appearing on the screen (those were white), with approx one frame delay after red ones from the RGB Mat. So there were both. How is this possible? I did not draw into mRGB mat any white rectangles :/

Here I call it in Java:

public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    final int viewMode = mViewMode;
    switch (viewMode) {        
       case VIEW_MODE_FEATURES:
        // input frame has RGBA format
        mRgba = inputFrame.rgba();
        mGray = inputFrame.gray();

        myDetect();//native function
        //Imgproc.cvtColor(mGray, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);

        break;
    }

    return mRgba;
}

And this is how it looks like in C++ part

    Mat& mGr = *(Mat*) addrGray;
    Mat& mRgb = *(Mat*) addrRgba;

    vector<Rect> highLight; // vector with rectangles to highlight detected items

    detect(mGr, highLight, env, stages);

    //highlight detected items on screen 
    for (unsigned int i = 0; i < highLight.size(); i++) {
        rectangle(mRgb, highLight[i], Scalar(255, 0, 0, 255),THICKNESS);
        //rectangle(mGr, highLight[i], Scalar(255, 0, 0, 255),THICKNESS);
          // if I leave this here uncommented, white rectangle ghost will appear on my screen both with desired red ones...
}

Anybody could guess how is this possible? Thanks in advance!

Answer 1

Why would you be surprised?. If you look at the code carefully the input frame is the same but you are only supplying different channels i.e inputframe.rgba returns 4 channels and inputframe.gray returns single channel of the same input frame. It draws that white rectangle because in gray channel it can only draw different shades gray form bright to dark.

But for my surprise, those were still appearing on the screen (those were white), with approx one frame delay after red ones from the RGB Mat

This is because you draw the rectangle in grayscale mat after you draw it in RGBA frame. It appears to you as though its drawing one frame after.

rectangle(mRgb, highLight[i], Scalar(255, 0, 0, 255),THICKNESS);
rectangle(mGr, highLight[i], Scalar(255, 0, 0, 255),THICKNESS);

Have you wondered how you can detect those rectangles in grayframe

detect(mGr, highLight, env, stages);

and draw the detected results on rgba frame ?

rectangle(mRgb, highLight[i], Scalar(255, 0, 0, 255),THICKNESS);

Its the same reason as explained above.