Detecting shades of color in OpenCV

Question

I have a lots of images of paper cards of different shades of colors. Like all blues, or all reds, etc. In the images, they are held up to different objects that are of that color.

I want to write a program to compare the color to the shades on the card and choose the closest shade to the object.

however I realize that for future images my camera is going to be subject to lots of different lighting. I think I should convert into HSV space.

I'm also unsure of what type of distance measure I should use. Given some sort of blobs from the cards, I could average over the HSV and simply see which blob's average is the closest.

But I welcome any and all suggestions, I want to learn more about what I can do with OpenCV.

EDIT: A sample

Here I want to compare the filled in red of the 6th dot to see it is actually the shade of the 3rd paper rectangle.

Answer 1

I think one possibility is to do the following:

Color histograms from Hue and Saturation channels

1. compute the color histogram of the filled circle.
2. compute color histogram of the bar of paper.
3. compute a distance using histogram distance measures. Possibilities here includes: Chi square, Earthmover distance, Bhattacharya distance, Histogram intersection etc.

Check this opencv link for details on computing histograms

Check this opencv link for details on the histogram comparisons

Note that when computing the color histograms, convert your images to HSV colorspace as you yourself suggested. Then, there is 2 things to note here.

1. [EDITED to make this a suggestion rather than a must do because I believe V channel might be necessary to differentiate the shades. Anyhow, try both and go with the one giving better result. Apologies if this sent you off track.] One possibility is to only use the Hue and Saturation channels i.e. you build a 2D histogram rather than a 3D one consisting of values from the hue and saturation channels. The reason for doing so is that the variation in lighting is most felt in the V channel. This, together with the use of histograms, should hopefully make your comparisons more robust to lighting changes. There is some discussion on ignoring the V channel when building color histograms in this post here. You might find the references therein useful.
2. Normalize the histograms using the opencv functions. This is to account for the different sizes of the patches of material (your small circle vs the huge color bar has different number of pixels).

You might also wish to consider performing some form of preprocessing to "stretch" the color in the image e.g. using histogram equalization or an "S curve" mapping so that the different shades of color get better separated. Then compute the color histograms on this processed image. Keep the information for the mapping and perform it on new test samples before computing their color histograms.

Using ML for classification

Besides simply computing the distance and taking the closest one (i.e. a 1 nearest neighbor classifier), you might want to consider training a classifier to do the classification for you. One reason for doing so is that the training of the classifier will hopefully learn some way to differentiate between the different shades of hues since it has access to them during the training phase and is required to differentiate them. Notice that simply computing a distance, i.e. your suggested method, may not have this property. Hopefully this will give better classification.

The features use in the training can still be the color histograms that I mention above. That is, you compute color histograms as described above for your training samples and pass this to the classifier along with their class (i.e. which shade they are). Then, when you wish to classify a test sample, you likewise compute a color histogram and pass it to the classifier and it will return you the class (shade of color in your case) the color of the test sample belongs to.

Potential problems when training a classifier rather than using a simple distance comparison based approach as you have suggested is partly the added complexity of the program as well as potentially getting bad results when the training data is not good. There is also going to be a lot of parameter tuning involved to get it to work well.

See the opencv machine learning tutorials here for more details. Note that in the examples in the link, the classifier only differentiate between 2 classes whereas you have more than 2 shades of color. This is not a problem as the classifiers in general can work with more than 2 classes.

Hope this helps.