I'm sorry if this is a duplicate of some thread. I know there are lots of decompositions to decompose a matrix (like LU or SVD), but now I have an arbitrary non-square matrix and I want to decompose it to product of two matrices of given shape. If exact solution do not exist, I want to find a least-square one. If more then one solution exists, any of them would be fine.
I was using iterative methods like this:
A = np.random.rand(...)
B = np.random.rand(...)
for i in range(999):
A = np.linalg.lstsq(B.T, Y.T, None)[0].T
B = np.linalg.lstsq(A, Y, None)[0]
This is straightforward, but I found it converges sublinearly (actually logarithmicly), which is slow. I also found it sometimes (or frequently?) "bounces" back to a very high L2 loss. I'm wondering are there improvements exists to this or simply solving AB=Y should be done in a totally different way?
Thanks a lot!
You can do this with the SVD. See, for example the wiki article Suppose, for example you had an mxn matrix Y and wanted to find a factorisation
Y = A*B where A is mx1 and B is nx1
so that
A*B
is as close as possible (measured by the Frobenius norm) to Y.
The solution is to take the SVD of Y:
Y = U*S*V'
and then take
A = s*U1 (the first column of A, scaled by the first singular value)
B = V1' (the first column of V)
If you want A to be mx2 and B 2xn, then tou take the first two colums (for A scaling the first column by the first singular value, the second column by the second singular value), and so on.