Search code examples
rmemory-leaksconnectionstdin

Memory leak with closed connections

I have an R application interacting with a Java daemon via stdin and stdout in an infinite loop, which seems to have some memory leaks. Simplified R app:

while (TRUE) {
    con <- file('stdin', open = 'r', blocking = TRUE)
    line <- scan(con, what = character(0), nlines = 1, quiet = TRUE)
    close(con)
}

This loop ends up using more and more RAM, and even if I manually gc() after the close(con) call, the memory footprint seems to be OK for a while, but eventually grows forever.

A basic script to confirm this:

Rscript --vanilla -e "while(TRUE)cat(runif(1),'\n')" | Rscript --vanilla -e "cat(Sys.getpid(), '\n');while (TRUE) {con <- file('stdin', open = 'r', blocking = TRUE);line <- scan(con, what = character(0), nlines = 1, quiet = TRUE);close(con);gc()}"

This will start two R processes: one writing to stdout and the other reading from stdin connected with a pipe (and the second printing the pid so that you can monitor the related memory usage):

enter image description here

I'm not sure what I'm doing wrong, but would love to stop this memory leak so any help is highly appreciated.

Solution

  • Indeed (about reading about this on R-devel); notably, the memory leak has now been plugged in the development version of R, thanks to a patch by Gabor Csardi.