Search code examples
c++flutteropencvcolormatrix

Apply a 4x5 Matrix (instead of 4x4) to Transform in OpenCV (C++)

On Flutter side I use a 4x5 Matrix to apply a Color Filter to an image (for preview). Via FFI, I then apply the filter with OpenCV and transform on the image and save the new image result.
However, the result is always different to the preview.

So I was wondering, how can I achieve the same result with the same Matrix (4x5) with OpenCV?

Flutter - ColorFilter Matrix (RGBA) for ColorFiltered Widget:

const ColorFilter _FILTER_2 = ColorFilter.matrix(<double>[
  0.9,  0.11, 0.11, 0.0,  0.0, //
  0.11, 0.7,  0.44, 0.0,  0.0, //
  0.22, 0.22, 0.9,  0.0,  0.0, //
  0.0,  0.0,  0.0,  1.0,  0.0
]);

C++ - transform function with matrix (BGRA):

__attribute__((visibility("default"))) __attribute__((used)) void filter_2(char *inputImagePath, char *outputImagePath)
{
    Mat input = imread(inputImagePath);
    Mat output = input.clone();

    Mat filter = (Mat_<float>(4, 4) <<
                  0.22, 0.22, 0.9,  0.0,
                  0.11, 0.7,  0.44, 0.0,
                  0.9,  0.11, 0.11, 0.0,
                  0.0,  0.0,  0.0,  1.0);

    transform(output, input, filter);
    imwrite(outputImagePath, input);
}

Results

Filter Result Comparison

Solution

  • The most common channel order of color images is RGB. OpenCV's native channel order is BGR (the reverse) for historical reasons.

    // maps RGB to RGB
Mat filter = (Mat_<float>(4, 4) <<
    0.9,  0.11, 0.11, 0.0
    0.11, 0.7,  0.44, 0.0
    0.22, 0.22, 0.9,  0.0
    0.0,  0.0,  0.0,  1.0);

    Your color matrix for mapping RGB to RGB (with some effect) has certain values in the top left 3x3 part, which combine the RGB values in new ways.

    To account for the RGB/BGR difference in the output data, I see that you permuted rows of your mixing matrix. That matrix would correctly map RGB input to BGR output.

    // maps RGB to BGR (rows permuted)
Mat filter = (Mat_<float>(4, 4) <<
    0.22, 0.22, 0.9,  0.0
    0.11, 0.7,  0.44, 0.0
    0.9,  0.11, 0.11, 0.0
    0.0,  0.0,  0.0,  1.0);

    You could use that matrix, but use cvtColor on the input (with COLOR_BGR2RGB), to make it suitable for that matrix.

    Or you could also permute the columns of the matrix, so its input is assumed to be BGR:

    // maps BGR to BGR (columns and rows permuted)
Mat filter = (Mat_<float>(4, 4) <<
    0.9,  0.22, 0.22, 0.0,
    0.44, 0.7,  0.11, 0.0,
    0.11, 0.11, 0.9,  0.0,
    0.0,  0.0,  0.0,  1.0);

    In the application of this matrix, an input pixel's color channels are taken as a column vector (and a 1 is appended to enable affine transformations). This vector is then multiplied as M v, so the coefficients in the first column are multiplied by the first color component (for each output component), and the first row mixes all components of the input pixel into the first component of the output pixel.