How to get uncertainty/covariance in estimating the pose of ArUco marker - OpenCV

I am using OpenCV's cv2.aruco.estimatePoseSingleMarkers function to estimate the pose of an ArUco marker & it works well. However, I also want the uncertainty / covariance in the pose estimate.

Is there a way to achieve this?

Solution

I too have been searching for papers on uncertainty in pose. I have not been very successful, but have found several papers that theoretically examine issues around this... difficult to apply in practice. One paper even said something to the effect "nobody is thinking about error" LOL

I just gave up on my literature search and am attempting the following approach:

Created a group of four ArUco tags in a square. This gives four estimates centered around a "group center". Get four poses and associated data (rvecs, tvecs)
Using cv2.projectPoints, find group center four different ways (from each pose). This gives 2D (pixel space) random error. Kind of useful - you can get an estimate of local pixel precision (mm/pixel).
Using 3D rotation/translation (i.e. build and use the 4×4 transformation matrix, which requires cv2.Rodrigues), find group center in camera spacefour different ways - I think this should give 3DOF (translational) random error. It does give mean±std of the group center location, from std one can get SE (standard error) -> propagate error if needed to other locations.
Finally, again using 3D rotation/translation, I projected the Z-axis of each pose into camera space. Unit Z gives pose direction cosines, and again, mean±std gives random error of orientation. This would be the other 3DOF (rotational) random error I hope...

This is just random error, though. Now I'm thinking about how to include precision error... I usually fall back to "An Introduction to Error Analysis" by John R Taylor (the "Train Wreck" book)... he says

...For example, if you have to measure a clearly defined length l with a ruler graduated in millimeters, you might reasonably decide that the length could be read to the nearest millimeter but no better. Here, the uncertainty dl would be dl = 0.5mm...

Using the same (I think?) reasoning, I forgo subpixel refining and am going to begin a pixel precision estimate assuming pixel precision error is half a pixel. But surely there is someone smarter who can correct this? Donno.

I will update this as I progress...

Edit: a couple more thoughts.

cv.estimatePoseBoard of course does this too - which I think I read gives an "improved" estimate... but it doesn't return any uncertainty info! What a loss.
Once one has pose uncertainty, well, then you probably have to get it out to some other position. Franaszek and Cheok at NIST have published "Propagation of orientation uncertainty of 3D rigid object to its points" (I can't remember where I found the PDF) on how to do this - I haven't thoroughly incorporated this work into mine yet but it seems like (?) it would be useful for projecting uncertainty onto other parts of a rigid body on which the pose has been obtained.