Use MATLAB cameraParams in OpenCV program

I have a MATLAB program that loads two images and returns two camera matrices and a cameraParams object with distortion coefficients, etc. I would now like to use this exact configuration to undistort points and so on, in an OpenCV program that triangulates points given their 2D locations in two different videos.

function [cameraMatrix1, cameraMatrix2, cameraParams] = setupCameraCalibration(leftImageFile, rightImageFile, squareSize)
% Auto-generated by cameraCalibrator app on 20-Feb-2015

The thing is, the output of undistortPoints is different in MATLAB and OpenCV even though both use the same arguments.

As an example:

>> undistortPoints([485, 502], defaultCameraParams)
ans = 485   502

In Java, the following test mimics the above (it passes).

public void testUnDistortPoints() {
    Mat srcMat = new Mat(2, 1, CvType.CV_32FC2);
    Mat dstMat = new Mat(2, 1, CvType.CV_32FC2);
    srcMat.put(0, 0, new float[] { 485, 502 } );

    MatOfPoint2f src = new MatOfPoint2f(srcMat);
    MatOfPoint2f dst = new MatOfPoint2f(dstMat);

    Mat defaultCameraMatrix = Mat.eye(3, 3, CvType.CV_32F);
    Mat defaultDistCoefficientMatrix = new Mat(1, 4, CvType.CV_32F);

    Imgproc.undistortPoints(
            src,
            dst,
            defaultCameraMatrix,
            defaultDistCoefficientMatrix
    );

    System.out.println(dst.dump());

    assertEquals(dst.get(0, 0)[0], 485d);
    assertEquals(dst.get(0, 0)[1], 502d);
}

However, say I change the first distortion coefficient (k1). In MATLAB:

changedDist = cameraParameters('RadialDistortion', [2 0 0])
>> undistortPoints([485, 502], changedDist)
ans = 4.8756    5.0465

In Java:

public void testUnDistortPointsChangedDistortion() {
    Mat srcMat = new Mat(2, 1, CvType.CV_32FC2);
    Mat dstMat = new Mat(2, 1, CvType.CV_32FC2);
    srcMat.put(0, 0, new float[] { 485, 502 } );

    MatOfPoint2f src = new MatOfPoint2f(srcMat);
    MatOfPoint2f dst = new MatOfPoint2f(dstMat);

    Mat defaultCameraMatrix = Mat.eye(3, 3, CvType.CV_32F);
    Mat distCoefficientMatrix = new Mat(1, 4, CvType.CV_32F);
    distCoefficientMatrix.put(0, 0, 2f); // updated

    Imgproc.undistortPoints(
            src,
            dst,
            defaultCameraMatrix,
            distCoefficientMatrix
    );

    System.out.println(dst.dump());

    assertEquals(4.8756, dst.get(0, 0)[0]);
    assertEquals(5.0465, dst.get(0, 0)[1]);
}

It fails with the following output:

[0.0004977131, 0.0005151587]

junit.framework.AssertionFailedError: 
Expected :4.8756
Actual   :4.977131029590964E-4

Why are the results different? I thought Java's distortion coefficient matrix includes both the radial and tangential distortion coefficients.

Also, is CV_64FC1 a good choice of type for the camera / distortion coefficient matrices?

I was trying to test the effect of changing the camera matrix itself (i.e. the value of f_x), but it's not possible to set the 'IntrinsicMatrix' parameter when using cameraparams, so I want to solve the distortion matrix problem first.

Any help would be greatly appreciated.

Solution

There is a couple of things you have to take into account when working with calibration models.

First, note there exist several camera calibration and distortion models: Tsai, ATAN, Pinhole, Ocam. I assume you want to use the Pinhole model, which is the used by OpenCV and the most common one. It models from 2 to 6 parameters for radial distortion (denoted as k1...k6) and 2 for tangential distortion (denoted as p1, p2), as you can read in the OpenCV doc. Bouget's calibration toolbox for Matlab uses this model too.

Second, there is not a standardized way to arrange distortion parameters in a vector. OpenCV expects items in this order: [k1 k2 p1 p2 k3...k6], being k3...k6 optional.

So, check the documentation of your Matlab calibration software and look for what model it uses and in which order the parameters are arranged. Then, make sure it meets the order in OpenCV.

The calibration parameters for OpenCV are ok as CV_32F and CV_64F as I recall.

Update

I don't know in Java, but in C++, when you create a Mat, its initial values are unspecified, so that this code may be creating a matrix with a 2f in the first item and garbage in the remaining ones:

Mat distCoefficientMatrix = new Mat(1, 4, CvType.CV_32F);
distCoefficientMatrix.put(0, 0, 2f);

Could you check if this is the problem?

A note for the future, to make things trickier, take into account that the intrinsic calibration matrix in OpenCV is the transpose of the one in Matlab.