The Problem
I'm trying to use Blender to create synthetic image for use with OpenCV's pose estimation (specifically, OpenCV's findEssentialMat
and recoverPose
). However, I found that the rotation matrix R that OpenCV returns corrects for rotations along the camera's Y and Z axes, but not its X axis. I suspect this is because Blender and OpenCV have different camera models (see diagram), but I can't figure out how to correct for this. How would I take a rotation matrix made using OpenCV's camera model and apply it to Blender's camera model?
Additional Details
To test this, I rendered a scene from (0, 0, 30) using an identity camera rotation, then rotated the camera by 10 degrees along X, Y and Z. First, here's the identity rotation matrix (no rotation):
Here's a 10-degree rotation around X:
And here's a 10-degree rotation around Y:
Finally, here's a 10-degree rotation around Z:
Applying the rotation matrix OpenCV returns (for the estimated transformation between the rotated image and the original) corrects for all of these rotations except around X, which looks like this:
It seems that instead of correctly rotating by -10 degrees around X, the rotation matrix rotates a further 10 degrees around X.
Going from an OpenCV matrix to Blender could be accomplished by multiplying by another rotation matrix to compensate for the change in coordinates systems.
mat = [[1, 0, 0],
[0, -1, 0],
[0, 0, -1]]
You said the Y and Z components are already being compensated for so perhaps the negative of the matrix is what you need?