Webcam Image Processing in C++ OpenCV is Slow

Question

I want to simulate protanopia view (one of the partial colorblind) with C++ OpenCV using webcam to simulate it. Here the code:

#include "opencv2/opencv.hpp"

using namespace cv;

int main(int, char**)
{
    int i = 0;
    Mat im, im2, kernel3;
    Mat rgb2lms = (Mat_<double>(3, 3) << 17.8824, 43.5161, 4.11935, 3.45565, 27.1554,   3.86714, 0.0299566, 0.184309, 1.46709); //filter1
    Mat lms2lmsp = (Mat_<double>(3, 3) << 0, 2.02344, -2.52581, 0, 1, 0, 0, 0, 1); //filter2
    Mat Result, Result2, op1, op2, op3, op4, op5, op6;
    Vec3b zay, zay2;
    kernel3 = rgb2lms.inv(DECOMP_LU);       //filter 3
    cv::Mat mat(3, 1, CV_64FC1);     //create MAT for matrices multiplication 

Mat frame;
VideoCapture cap(0); // open the default camera
if (!cap.isOpened())  // check if we succeeded
    return -1;
namedWindow("edges", 1);
for (;;)
{

    cap.read(frame); // get a new frame from camera
    if (frame.empty()) continue;
    const int nChannels = frame.channels();
    if (i == 0){
        Result.create(frame.size(), frame.type());
    }
  //Result2.create(frame.size(), frame.type());
    cvtColor(frame, im2, CV_BGR2RGB);   //convert to RGB
    for (int i = 0; i < im2.rows; i++)
    {
        for (int j = 0; j < im2.cols; j++)
            {
                for (int k = 0; k < nChannels; k++)
                    {
                        zay(k) = im2.at<Vec3b>(i, j)[k]; //acces pixel value and put into 3x1 vector zay
                    //put the value in to mat so i can multiplied with easy
                    mat.at <double>(0, 0) = zay[0]; 
                    mat.at <double>(1, 0) = zay[1];
                    mat.at <double>(2, 0) = zay[2];
                    op1 = rgb2lms*mat;     //apply filter1
                    op2 = lms2lmsp*op1;    //apply filter2
                    op3 = kernel3*op2;     //apply filter3

                for (int k = 0; k < nChannels; k++)
                    {
                    Result.at<Vec3b>(i, j)[k] = op3.at<double>(k, 0);   //put the result from vector to mat
                    }

            }
    }

    cvtColor(Result, Result2, CV_RGB2BGR);  //convert back to BGR
    imshow("hasil", Result2);
    if (waitKey(30) >= 0) break;
    i++;
}
// the camera will be deinitialized automatically in VideoCapture destructor
return 0;

}

This code is run, but the image video output is very slow (lagging). i'm very new using C++. My Question:

1. Very slow output because i'm using many iterations?
2. I have tried this process using filter2D (convolution) and the result is different. Is the convolution different with matrix multiplication filter?
3. How to make the output smoothly? Should i use pointer in that code?

Thanks

Answer 1

Filter2D is not the appropriate function for your task. It applies a convolution filter on each channel of the image, while what you want is a linear tranformation matrix applied on each rgb pixel.

The function you are looking for is transform.

First, for better performance, merge your three transformations into a single global transformation matrix:

Mat global_kernel = kernel3*lms2lmsp*rgb2lms;

Then, instead of your for loop, use:

transform(im2, Result, global_kernel);

If you still want to save a few milliseconds, you may also remove the cvtColor function calls by applying your transformation directly from bgr color space. Simply switch the columns of your rgb2lms matrix:

Mat bgr2lms = (Mat_<double>(3, 3) << 4.11935, 43.5161, 17.8824,  3.86714, 27.1554, 3.45565,  1.46709, 0.184309, 0.0299566);