Undistorting an image with corresponding points in Python

Question

I have an image with specific rectangles on it. I want to undistort these rectangles so I can use the pixel distances to estimate real-world distances.

I started out using code I found online, but the resulting images don't seem to match up. Normal:

Undistorted image:

Where the blue rectangles are always normal and green is undistorted. I also tried using the initUndistortRectifyMap and initInverseRectificationMap function to remap both the image and points with greater success, but one rectangle is still being mapped in a weird way:

Code

General:

img = cv2.imread(str(image))
IMG_SIZE = img.shape[:-1]  # h, w
optimal_camera_matrix, roi = cv2.getOptimalNewCameraMatrix(camera_matrix, dist_coeffs, IMG_SIZE[::-1], 1, IMG_SIZE[::-1])
maps = cv2.initUndistortRectifyMap(
        camera_matrix,
        dist_coeffs,
        None,
        optimal_camera_matrix,
        IMG_SIZE[::-1],
        cv2.CV_32FC1,
    )
inv_maps = cv2.initInverseRectificationMap(
        camera_matrix,
        dist_coeffs,
        None,
        optimal_camera_matrix,
        IMG_SIZE[::-1],
        cv2.CV_32FC1,
    )

First version:

undistorted_image = cv2.undistort(img, camera_matrix, dist_coeffs)
undistorted_begin = cv2.undistortPoints(begin, camera_matrix, dist_coeffs, None, optimal_camera_matrix).squeeze()
undistorted_end = cv2.undistortPoints(end, camera_matrix, dist_coeffs, None, optimal_camera_matrix).squeeze()

Second version:

undistorted_image = cv2.undistort(img, camera_matrix, dist_coeffs, None, optimal_camera_matrix)
undistorted_image = cv2.remap(img, maps[0], maps[1], cv2.INTER_LINEAR)
undistorted_begin = cv2.undistortPoints(begin, camera_matrix, dist_coeffs, None, optimal_camera_matrix).squeeze()
undistorted_end = cv2.undistortPoints(end, camera_matrix, dist_coeffs, None, optimal_camera_matrix).squeeze()

Answer 1

undistort points image show

it seems work in my opencv cpp code. could u send me your data and parameters? l will try it.

int test_remap() {
std::string src_path = "./calib_data/left05.jpg";
cv::Mat intrinsic = LoadParams("./workspace/intrinsic.yaml").front();
cv::Mat distortion_coeff = LoadParams("./workspace/distortion_coeff.yaml").front();

cv::Mat test_img = cv::imread(src_path);
cv::Mat undistort_img;
cv::Mat map_x, map_y;
cv::Mat intrinsic_copy = intrinsic.clone();
cv::initUndistortRectifyMap(intrinsic, distortion_coeff, cv::Mat(), 
        intrinsic_copy, test_img.size(), CV_32FC1, map_x, map_y);
cv::remap(test_img, undistort_img, map_x, map_y, cv::InterpolationFlags::INTER_LINEAR,
        cv::BorderTypes::BORDER_CONSTANT, 0);

std::vector<cv::Point2f> undistort_pts, to_un{cv::Point2f(241., 97.)};
cv::undistortPoints(to_un, undistort_pts, intrinsic, distortion_coeff, cv::Mat(), intrinsic);
std::cout << std::endl;
std::cout << "undistort_pts " <<  undistort_pts.front() << std::endl;

cv::circle(test_img, to_un.front(), 3, cv::Scalar(0,0,255));
cv::circle(undistort_img, undistort_pts.front(), 3, cv::Scalar(0,255,0));

cv::Mat concat_img;
cv::hconcat(std::vector<cv::Mat>{test_img, undistort_img}, concat_img);
cv::imshow("undistort", concat_img);
cv::waitKey(0);}