I can't find clear definition for the built-in 4x4 matrices like Object2World or UNITYMATRIX_MVP regarding how they are structured(row-major or column-major) and that confuses me about where they should be put in the mul() function.
in cases like
o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
float3 viewDirection = normalize( float3( float4( _WorldSpaceCameraPos, 1.0) - mul(_Object2World, v.vertex) ) );
the UNITYMATRIXMVP and the _Object2World matrices are the first argument in the mul() function.
so it seems that the UNITYMATRIXMVP and the _Object2World should be row major according to the rule of Matrix-Vector left multiplication right?
And, the the second argument which is v.vertex should be a 4x1 vector, so is a float4 here automatically treated as a 4x1 but not a 1x4 vector?
in the following case
float3 normalDirection = normalize( mul( float4(v.normal, 0.0), _World2Object ).xyz );
the _World2Object is used as the second argument
so how does World2Object, UNITYMATRIX_MVP and other built-in matrices actually look like internally, are they row major? like:
X1,X2,X3,X4
Y1,Y2,Y3,Y4
Z1,Z2,Z3,Z4
W1,W2,W3,W4
or column major? like:
X1,Y1,Z1,W1
X2,Y2,Z2,W2
X3,Y3,Z3,W3
X4,Y4,Z4,W4.
All I can do now is doing guessing by trial and error. Are those built-in matrices row-major or column-major? is there any clear documentation about it?
Many Thanks~~
Matrices in Unity are column-major. I.e. you must multiply matrix by vector to apply affine transformation provided by matrix to the given vector.
I understand your confusion with vector and matrix products. The fact is matrix products are mathematically fair, but matrix by vector (and reverse) multiplications aren't. When you multiply vector by matrix (or matrix by vector), mathematically it is still multiplication of matrices.
Remember that you can multiply (N x M) matrix and (M × P) matrix? Thats it. You may multiply (1x4) by (4x4) or (4x4) by (4x1) (but not (4x1) by (4x4) and not (4x4) by (1x4)). So, when you are multiplying vector and matrix you must consider vector as a matrix - a single-row matrix (1x4) or a single-column matrix (4x1) and you must multiply them in a right order. That's why when you are using post-multiplication M.V you must have matrix in a column-major order to receive a result that makes a sense.