I had the following question, which is similar: Doesn't Matlab optimize the following?
but this is one is a harder variant, and the solution suggested there does not work.
I have a very long matrix n x r V, and a very long matrix W n x s , and a matrix A rxs, which is sparse (but very big in dimensions).
I was expecting the following to be optimized by Matlab so I won't run into trouble with memory:
A./(V'*W)
but it seems like Matlab is actually trying to generate the full V'*W matrix, because I am running into Out of memory issue. Is there a way to overcome this? Note that there is no need to calculate all V'*W because many values of A are 0.
If that were possible, one way to do it would be to do A(find(A)) ./ (V'*W)(find(A));
but you can't select a subset of a matrix (V'*W in this case) without first calculating it and putting it in a variable.
(The main difference from the previous question: V and W are not just vectors, but matrices.)
The answer to the previous question can be generalized to compute the nonzero values of A./(V'*W) as follows:
[ii jj Anz] = find(A);
result = arrayfun(@(n) Anz(n) / ( V(:,ii(n))'*W(:,jj(n)) ), 1:length(ii) );
This avoids computing the full matrix V'*W, because it only computes the needed entries of that matrix (i.e. those for which the corresponding entry of A is nonzero), one at a time. So memory use is kept low. As for speed, using arrayfun is usually slow, but if A has few nonzero values it should't take long.
Of course, you could then generate the sparse matrix A./(V'*W) as sparse(ii,jj,result).