Select specific range of elements in Combination vector in MATLAB

Question

I am creating a mesh grid or combination vector of an array. The full combination vector is so large that MATLAB prompts me "out of memory" error. I have written a code to create the mesh grid, depending on the variable "Col".

Col=9;
numMatrix = uint8([1 2 3]);
meshString=strcat("a3 = combvec(" ,repmat('numMatrix ,', 1, Col-1), "numMat )")
eval(meshString);

If the Col value is 9, then the size of the matrix formed is this.

Variable              Size               Byte
 a3                   9x19683            1417176  double

The problem with this code is that if my Col value is 25, then the size of a3 will be 25x3^25 which require the RAM of 253^258/1024/1024/1024 GB (1.5782e+05 GB) which is not realizable. So I want to create a specific range of combinations. For example, to have only the first 50000 combinations directly then after the processing goes for the next 50000.

P.S: I have also used the function [b1,b2....,b25]=ndgrid(1:3) to create the b's grids and then flattening them to form the combination. It also suffers from the same problem explained above.

Answer 1

The output of combvec is totally predictable so instead of generating the whole vector you could simply compute each iteration individually.

Let's take a simple example:

% we compute combvec(1:3,1:3,1:3,1:3)
n = 4;
x = 1:3;
x = repmat({x},n,1)
y = combvec(x{:})

Here y is a 4x81 matrix. And we noticed that sinced we only use combination of the vector [1,2,3] then the j-th column will correspond to j-1 in base 3.

So we can write a small function that will convert a decimal number in base 3:

function r = f(j,n)
    r = fliplr(dec2base(j-1,3,n)-47).' % small tricks with -47 to implicitely convert a char to its ascii value minus 47.
end

And now we can verify the equality, for example with j=10:

all(f(10,n) == y(:,10))

And we indeed got: True as a result.

It will also works for multiple value:

all(f(9:10,n) == y(:,9:10))

So you just need to add the last row of combvec and now you can split your operation and let run your computer for a week.