Search code examples
rsparse-matrix

Dealing with very large expand.grid?

I am given 31 binary variables, for simplicity let's call them x_1, ..., x_31.

Each X_i = c(0, 1).

I need to build a tree of options. I have used expand.grid for this but my server throws an error:

Error: cannot allocate vector of size 16.0 Gb.

I am aware that I can sparsify this but how can I deal with the large size?

Using sparsify gives the same error message but with 8 Gb.

Please advise.

Solution

  • Too long for a comment, and @steves asked me to elaborate my comment. Take e.g.

    > expand.grid(c(0,1), c(0,1), c(0,1))
  Var1 Var2 Var3
1    0    0    0
2    1    0    0
3    0    1    0
4    1    1    0
5    0    0    1
6    1    0    1
7    0    1    1
8    1    1    1

    This table has 2^3=8 lines. At line i, this is the binary expansion of i-1:

    > f <- function(i) as.integer(rev(intToBits(i)))
> f(6-1) # this gives line 6, with some heading 0's
 [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1

    For expand.grid(x1, ..., x31) there are 2^31 lines. The output of f(i) is a vector of 32 digits. So if you modify f by removing the first digit:

    f <- function(i) as.integer(rev(intToBits(i)))[-1]

    then f(i-1) exactly provides line i of expand.grid(x1, ..., x31).