Memory profiling in R - tools for summarizing

Question

R has some tools for memory profiling, like Rprofmem(), Rprof() with option "memory.profiling=TRUE" and tracemem(). The last one can only be used on objects, and hence is useful to follow how many times an object is copied, but doesn't give an overview on a function basis. Rprofmem should be able to do that, but the output of even the simplest function call like lm() gives over 500 lines of log. I tried to figure out what Rprof("somefile.log",memory.profile=T) actually does, but I don't think I really get it.

The last I could find was this message of Thomas Lumley, saying that, and I quote :

I do not yet have tools to summarize the output.

This was in 2006. Any chance there are options for some nice summaries now, based on either Rprofmem(), the mysterious output of Rprof() with memory.profile set TRUE or any other tool?

Answer 1

profvis looks like the the solution to this question.

It generates an interactive .html file (using htmlwidgets) showing the profiling of your code.

The introduction vignette is a good guide on its capability.

Taking directly from the introduction, you would use it like this:

devtools::install_github("rstudio/profvis")
library(profvis)

# Generate data
times <- 4e5
cols <- 150
data <- as.data.frame(x = matrix(rnorm(times * cols, mean = 5), ncol = cols))
data <- cbind(id = paste0("g", seq_len(times)), data)
profvis({
    data1 <- data   # Store in another variable for this run

    # Get column means
    means <- apply(data1[, names(data1) != "id"], 2, mean)

    # Subtract mean from each column
    for (i in seq_along(means)) {
        data1[, names(data1) != "id"][, i] <- data1[, names(data1) != "id"][, i] - means[i]
    }
}, height = "400px")

Which gives