Homography? OpenCV method to handle missing chessboard corners in some views?

Question

I have just installed OpenCV 3.2.0 and I can find many/most of the corners in this image from this question by playing around with the contrast then using cv2.findChessboardCorners(left_gray, (6,5)) or by using cv2.goodFeaturesToTrack(gray_img,25,0.01,10) as described in the answer.

While findChessboardCorners either returns all 6×5=30 of them or None, I can see that for lower quality images sometimes not all points are going to be found with the goodFeaturesToTrack method.

But when I read the documentation for cv2.calibrateCamera() it looks like the object points need to correspond on a one-to-one basis with the image points for each and every view. If one of the images has a missing point, I'd have to find it and remove it from the object point list for that image.

Is there some cv2 method to automatically handle this?

edit: I'm now wondering, is cv2.findHomography() using one of the robust methods what I'm looking for here?

Of course if an external row or column is all missing, there is an unresolvable ambiguity, but if a few internal points are missing, a calibration should still be possible.

I could probably try to write something, and of course just reject any view with the wrong number of found corners, but if there is already an existing way to handle non-ambiguity-causing missing points, I'd like to try it first. The goal would be to automate the process as much as possible without skipping every image with the wrong number of found points.

Answer 1

You could handle this problem by slightly changing pattern being used. What you need here is to specify which corners are missing. OpenCV has a simple solution for that: ChArUco board. Below I attached tutorial of that pattern usage.

http://docs.opencv.org/3.1.0/df/d4a/tutorial_charuco_detection.html

With usage of charuco you can specify which corners are detected - automatically. Then you can adjust object points for calibration.

If you are using python you need wrapper of aruco library:

https://github.com/opencv/opencv_contrib/pull/554/

https://github.com/fehlfarbe/python-aruco/

And some answer for using charuco in python:

http://answers.opencv.org/question/98447/camera-calibration-using-charuco-and-python/?answer=98451#post-id-98451/

---

Briefly, from the linked tutorial, the process is as follows:

To define a CharucoBoard, it is necessary to generate the pattern using a definition, then specify the description to the detection algorithm as well:

• Number of chessboard squares in X direction.
• Number of chessboard squares in Y direction.
• Length of square side.
• Length of marker side.
• The dictionary of the markers.
• Ids of all the markers.