Search code examples
opencvcomputer-visionemgucvaforge

Finding an icon in an image

I'm just starting to learn about computer vision and am working on a simple project to find basic icons in a still image.

I have a template image: enter image description here

and two test images:

enter image description here and enter image description here

I used template matching (using AForge.net, but I think it is the same algorthym that OpenCV and Emgu use. I could be wrong, I'm new at CV) and discovered that with a threshold of .80563 that I would find exact one match in both of the above, and get no matches in images I tried that the icon was not in.

I thought I was getting somewhere until I looked at what was returned as the match in each image: (The blue highlighted squares are where the image was matched.)

enter image description here (Correct) and enter image description here (Completely wrong)

I realize that the issue is any or all of:

  • Icons I am looking for will be similar in size, color and shade, but no guarantee they will be identical in any of the above. (Note: Angular orientation and proportions should be consistent. I wouldn't want a circle with a "+" in it to match the template. Nor would I want a huge circle with a tiny "X" to match.)
  • My template isn't a square, and I'm just guessing that the transparent pixels will not be included in the match.
  • Template matching may be the wrong approach given the 2 issues above. Maybe I need to use something else to do this?

Ultimately, I need some basic help on what is going wrong with my matching so that I can at least get back moving in the correct direction. Is template matching the right approach, but I need to change something? Or do I need to look at one of the other capabilities in these libraries? For this simple task, is there much functional difference between OpenCV (and EMGU) functionality and AForge.net functionality?

Solution

  • Template matching is the right choice in this case, but you must do it right:

    • Your template is wrong for what you're looking for. In your second image you have bad results because the gray values of the button are more similar to some place in the background. If you want to search for the white cross, then use a white cross as the tamplate, like this:

    enter image description here

    • Don't use fixed thresholds, but search for the highest response points.

    With these guidelines, you can find your button (small red rectangle):

    enter image description here

    enter image description here

    This is the code in OpenCV (C++) as example:

    #include <opencv2\opencv.hpp>
using namespace cv;

int main()
{
    // Load template and image
    Mat3b templ = imread("path_to_template");
    Mat3b img = imread("path_to_image");

    // Convert to grayscale
    Mat1b img_gray;
    Mat1b templ_gray;
    cvtColor(img, img_gray, COLOR_BGR2GRAY);
    cvtColor(templ, templ_gray, COLOR_BGR2GRAY);

    // Match template
    Mat1f result;
    matchTemplate(img, templ, result, TM_CCOEFF_NORMED);

    // Find highest response
    Point maxLoc;
    minMaxLoc(result, NULL, NULL, NULL, &maxLoc);

    // Draw the red rectangle
    rectangle(img, Rect(maxLoc, templ.size()), Scalar(0,0,255), 2);

    // Show results
    imshow("Result", img);
    waitKey();

    return 0;
}