OpenCV - Are Lens Distortion Coefficients inverted for projectPoints?

Question

Good day guys,

I am trying to simulate and image with lens barrel distortion. I create a virtual chessboard (only the corners) and then project it onto my image plane using OpenCV. The idea is to project these points with known distortion coefficients, and then attempt a lens distortion calibration (with calibrateCamera), and see if the same coefficients are obtained.

My question is about the projectPoints function which takes distCoeffs as an input. Are these coefficients the same that must be used to undistort an image (output of calibrateCamera)? This means the function will have to calculate the inverse of that operation. Or, does it use those coefficients to distort the object points directly? Meaning that the will not correlate at all at the output of e.g. calibrateCamera.

I ask, because I noticed my simulation does pincushion distortion when I expect barrel, and vica versa. Which seems that the distortion does the opposite of what I think it does.

The minimal working code that I used to simulate the image (in Python):

# The distortion matrix that I vary
distortion = np.array([0.3, 0.001, 0.0, 0.0, 0.01])

# Generate Grid of Object Points
grid_size, square_size = [20, 20], 0.2
object_points = np.zeros([grid_size[0] * grid_size[1], 3])
mx, my = [(grid_size[0] - 1) * square_size / 2, (grid_size[1] - 1) * square_size / 2]
for i in range(grid_size[0]):
    for j in range(grid_size[1]):
        object_points[i * grid_size[0] + j] = [i * square_size - mx, j * square_size - my, 0]

# Setup the camera information
f, p = [5e-3, 120e-8]
intrinsic = np.array([[f/p, 0, 0], [0, f/p, 0], [0, 0, 1]])
rvec = np.array([0.0, 0.0, 0.0])
tvec = np.array([0.0, 0.0, 3.0])

# Project the points
image_points, jacobian = cv2.projectPoints(object_points, rvec, tvec, intrinsic, distortion)

# Plot the points (using PyPlot)
plt.scatter(*zip(*image_points[:, 0, :]), marker='.')
plt.axis('equal')
plt.xlim((-4000, 4000))
plt.ylim((-4000, 4000))
plt.grid()
plt.show()

Additional Explanation:

My query is about the wrong distortion type being created. If I use a positive distortion matrix, then I would expect barrel distortion, according to this website, which states:

The next figure shows two common types of radial distortion: barrel distortion (typically k_1 > 0) and pincushion distortion (typically k_1 < 0).

To test this I used the following positive distortion matrix (as in the code above), and it created pincushion distortion.

distortion = np.array([0.3, 0.001, 0.0, 0.0, 0.01])

In this image I clearly created pincushion distortion, and not barrel distortion.

Similarly, if I make my distortion coefficient negative, which should result in pincushion distortion, I get the following:

distortion = -np.array([0.3, 0.001, 0.0, 0.0, 0.01])

Does this mean that the distortion matrix is applied negatively if you use the projectPoints function?

Thank you for your help!

Answer 1

It seems that the distCoeffs in projectPoints are of the same type that calibrateCamera outputs. I ran a more thorough simulation where I added distortion using projectPoints, and then estimated it again using calibrateCamera, and it gave me the same distortion matrix.

E.g with the following code

distortion_given = np.array([[0.2, 0.01,  0.0, 0.0, 0.01]])# Note the negative sign
... = projectPoints(... , distortion_given , ...)
... , distortion_estimated , ... = calibrateCamera (...)
print (distortion_given)
print (distortion_estimated)

which created the following projected image with Barrel distortion

Resulted in:

distortion_given = [[-0.2, -0.01,  0.0, 0.0, -0.01]]
distortion_estimated = [[-0.19999985 -0.01000031  0.          0.         -0.00999981]]

This means that when this website I mentioned in the question should put more emphasis on the word typically, as I observed the opposite.

barrel distortion (typically k_1 > 0) and pincushion distortion (typically k_1 < 0).