Search code examples
c++opencvimage-processinghsv

Multiplication of 2 images where the images have different models in OPEN CV


I'm trying to multiply two images of different models, in my case HSV and YCRCB.
I get the "vector is out of bound error" every time. I have checked the sizes of the input images being multiplied, the number of rows and columns. I know the value is exceeding over 255.
I tried to implement this method opencv - image multiplication, but the code has way to many MAT's that have to be initialized. This also leads me to ask the question if images with more than 1 channel can be multiplied. Also tried direct multiplication and it doesn't work, so tried multiplying channel wise. To make things easier, I used the loop method but then the error occured.

A short summary about the code and reason for doing it : I'm using it for skin detection but want to further reduce noise. I think this can be done by multiplying the 2 output images generated by the threshold operations (for HSV & YCRCB). Since these images have different noises in the image, the output of the multiplication will have even less noise (I have seen the output on different screens, the overlapping regions are very small) hence this can detect skin color at all almost times and noise will be minimal and thus will help in tracking skin better.

The code given below is not complete cause it never executes till the end. After this there are morphological and dilation operations being done, that's it.
This is my first time asking a question on Stack Overflow and I'm still learning Open CV . Sorry If I have been over-descriptive and all suggestions are welcome. Thank You.

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv\cv.h>
#include <opencv\highgui.h>
#include <iostream>
#include <opencv2\imgproc\imgproc.hpp>

    using namespace cv;
    using namespace std;
    char key;
    Mat image,hsv,ycr;
    vector<Mat> channels,ycrs,threshold_output;   

     int main()
        {       
        VideoCapture cap(0); // open the default camera
            if(!cap.isOpened())  // check if we succeeded
                {
                 cout << "Cannot open the web cam" << endl;
                 return -1;
            }
        while(1)
        {

            cap>>image;
              cvtColor( image, ycr, CV_BGR2YCrCb ); //Converts into YCRCB 
              cvtColor( image, hsv, CV_BGR2HSV ); //Converts into HSV 
              Mat imgThresholded;
              Mat imgThresholded1;
              inRange(ycr, Scalar(0, 140,105 ), Scalar(255, 165,135), imgThresholded1); //for yrcrcb range
              inRange(hsv, Scalar(0, 48,150 ), Scalar(20, 150,255), imgThresholded); //for hsv range

    split(imgThresholded1, channels);
    split(imgThresholded, ycrs);
    for( int i = 0; i <3 ; i++ )
    {
        multiply(channels[i],ycrs[i], threshold_output[i], 1,-1 );
    }//code breaks here 

Solution

  • Even if the input to inRange are multi-channeled, the output of inRange will be a single-channel CV_8UC1.

    The reason is that inRange computes a Cartesian intersection:

    • Result (x, y) is true (uchar of 255) if ALL of these are true:
      • For first channel, lower[0] <= img(x, y)[0] <= upper[0], AND
      • For second channel, lower[1] <= img(x, y)[1] <= upper[1], AND
      • And so on.

    In other words, after it has checked each channel's pixel values against the lower and upper bound, the logical result is then "boiled down" Logical-And operation over the channels of the image.

    "Boiled down" is my colloquial way of referring to reduction, or fold, where a function can accept arbitrary number of arguments and it can be "reduced" down to a single value. Summation, multiplication, string concatenation, etc.

    It is therefore not necessary to use cv::split on the output of cv::inRange. In fact, because the output has only one channel, calling channels[1] or ycrs[1] will be an undefined behavior, which will either cause an exception for a debug-build and undefined behavior or crash or memory corruption for a release-build.