Perfect object to be recognized with OpenCV

Question

I have an application where I want to track 2 objects at a time that are rather small in the picture. This application should be running on Android and iPhone, so the algorithm should be efficient.

For my customer it is perfectly fine if we deliver some patterns along with the software that are attached to the objects to be tracked to have a well-recognizable target.

This means that I can make up a pattern on my own. As I am not that much into image processing yet, I don't know which objects are easiest to recognize in a picture even if they are rather small. Color is also possible, although processing several planes separately is not desired because of the generated overhead.

Thank you for any advice!! Best,

guitarflow

Answer 1

If I get this straight, your object should:

1. Be printable on an A4
2. Be recognizeable up to 4 meters
3. Rotational invariance is not so important (I'm making the assumption that the user will hold the phone +/- upright)

I recommend printing a large checkboard and using a combination of color-matching and corner detection. Try different combinations to see what's faster and more robust at difference distances.

Color: if you only want to work on one channel, you can print in red/green/blue*, and then work only on that respective channel. This will already filter a lot and increase contrast "for free". Otherwise, a histogram backprojection is in my experience quite fast. See here.

Also, let's say you have only 4 squares with RGB+black (see image), it would be easy to get all red contours, then check if it has the correct neighbouring colors: a patch of blue to it's right and a patch of green below it, both of roughly the same area. This alone might be robust enough, and is equivalent to working on 1 channel since for each step you're only accessing one specific channel (search for contours in red, check right in blue, check below in green).

If you're getting a lot of false-positives, you can then use corners to filter your hits. In the example image, you have 9 corners already, in fact even more if you separate channels, and if it isn't enough you can make a true checkerboard with several squares in order to have more corners. It will probably be sufficient to check how many corners are detected in the ROI in order to reject false-positives, otherwise you can also check that the spacing between detected corners in x and y direction is uniform (i.e. form a grid).

Corners: Detecting corners has been greatly explored and there are several methods here. I don't know how efficient each one is, but they are fast enough, and after you've reduced the ROIs based on color, this should not be an issue. Perhaps the simplest is to simply erode/dilate with a cross to find corners. See here . You'll want to first threshold the image to create a binary map, probably based on color as metnioned above. Other corner detectors such as Harris detector are well documented.

Oh and I don't recommend using Haar-classifiers. Seems unnecessarily complicated and not so fast (though very robust for complex objects: i.e. if you can't use your own pattern), not to mention the huge amount of work for training.