I'm trying to figure out how to get the equivalent of an arbitrary affine 3D matrix using only translation, rotation and non-uniform scaling.
Handling shearing is the tricky part. A single shear transformation can be expressed as a combination of rotation, non-uniform scale, and rotation as discussed here: Shear Matrix as a combination of basic transformation?
However, for 3D there can be shearing on multiple planes at once; for example XY, XZ, and YZ. While I could express each of those with rotation,scale,rotation, that would be a total of 6 rotations and 3 scaling operations. I have an intuition that all the shearing can be handled at once with just one rotation, non-uniform scale, and rotation, but the math involved is over my head.
I'm not sure what constitutes shearing versus rotation when looking at an arbitrary affine matrix (I think there's infinite solutions for how this is split up?) so I guess solving the issue for "arbitrary sharing along multiple planes" is the same as solving for just affine matrices (without translation) in general. Either way, anything that can help me along the way is appreciated.
A (full) SVD gets you close. This gives, for a 3x3 matrix A
A = U*S*V'
where all the matrices are 3x3, S is diagonal and U and V are orthogonal. Unfortunately U and V may not be rotations, that is, they might have determinant -1.
One way to proceed would be to compute the determinant of U, and if it is -1 to replace it with
U~ = U * diag(-1,1,1) (ie negate the first col of U)
and replace S with
S~ = S*diag( -1, 1, 1) (ie negate the top left elt of S)
And then similarly for V (though now, because of the transpose you want to negate the first row of V)