I'm taking a large (dense) network matrix and converting it to an edgelist. Yet, when I do so, the memory allocated in R seems crazy. In my case, I have a 12MB matrix (1259 x 1259) that when converted to an edgelist (i, j, w) is taking up 71MB of memory! I'm using the igraph package to perform the operations, but I don't think it is related to that. Here is what I'm doing with made up data.
library(igraph)
A <- matrix(runif(25), 5, 5)
A <- A %*% t(A)
diag(A) <- 0
I made the matrix symmetric and diagonal 0 because that is what my data looks like, but I don't think it matters for this question. Here I use igraph:
# using igraph here
adj <- graph.adjacency(as.matrix(A),weighted=TRUE)
object.size(A) # 400 bytes
object.size(adj) # 2336 bytes
I get that the igraph adj object will be bigger. That isn't the issue.
el <- get.edgelist(adj)
class(el) # "matrix"
object.size(el) # 520 bytes
w <- E(adj)$weight
class(w) # "numeric"
object.size(w) # 200 bytes
# expect something ~720 bytes
adj_w <- cbind(el,w)
class(adj_w) # "matrix"
object.size(adj_w) # 1016 bytes
Why is the memory on adj_w so much larger? It doesn't even seem to be linear since here, the original to final is 400 bytes to 1016 bytes but in my (bigger) data it is 12MB to 71MB.
FYI: I'm using RStudio locally on a Macbook Pro with the latest versions (just installed it all last week).
adj_w is larger because cbind added a column name. Remove it and you're back to the correct size.
head(adj_w)
# w
# [1,] 1 2 1.189969
# [2,] 1 3 1.100843
# [3,] 1 4 0.805436
# [4,] 1 5 1.001632
# [5,] 2 1 1.189969
# [6,] 2 3 1.265916
object.size(adj_w)
# 1016 bytes
attributes(adj_w)
# $dim
# [1] 20 3
#
# $dimnames
# $dimnames[[1]]
# NULL
#
# $dimnames[[2]]
# [1] "" "" "w"
#
#
adj_w2 <- adj_w
dimnames(adj_w2) <- NULL
object.size(adj_w2)
# 680 bytes
To avoid the automatic column name addition, you can first convert your vector to a matrix...
adj_w3 <- cbind(el, matrix(w))
object.size(adj_w3)
# 680 bytes
...or, alternatively, pass the deparse.level = 0 argument to cbind.
adj_w4 <- cbind(el, w, deparse.level = 0)
object.size(adj_w4)
# 680 bytes