I have a coordinate system in which the orientation of a camera is represented as R=Rz(k) * Ry(p) * Rx(o) where R is a 3x3 matrix of the composition of 3x3 rotation matrices around each of X,Y,Z-axis. Moreover, I have a convention in which Z-axis is in the viewing direction of the camera. The X-axis is left-right and the Y-axis is bottom-up.
This R matrix is used in a multi-view stereo reconstruction algorithm. I have a data test set which comes with pre-calibrated camera information. I want to use the R matrices that come with this data set. However, I have no idea what kind of rotation order they assume or even their handed-ness.
How would I be able to figure this out? Any ideas?
R=Rz(k) * Ry(p) * Rx(o)
This is a very instable way of doing it. Euler angles are prone to go into gimbal lock, so I strongly advise against their use.
How would I be able to figure this out?
Well, this problem is difficult to express in a closed solution. Your best bet is treating this as a optimization problem in 3 space, where you try find the values for k, p and o to match up with the given rotation matrix R. There are 3 possible permutations on the evaulation order, so you do that optimization for all 3 of them and take the best matching result.