Search code examples
rforeachdoparallel

Running foreach without returning any value in R

I have a function doSomething() which runs in a foreach loop and as a result saves some calculations as .csv files. Hence I have no need for a return value of foreach, in fact I don't want a return value because it clutters my memory to the point where I cannot run as many iterations as I would want to.

How can I force foreach to not have a return value, or delete the return values of the iterations?

Here is a minimal example that illustrates my problem:

cl <- parallel::makePSOCKcluster(1)
doParallel::registerDoParallel(cl)

"%dopar%" <- foreach::"%dopar%"

doSomething <- function () {
  a <- as.numeric(1L)
}

foreach::foreach (i = 1:4) %dopar% {

  doSomething()

}

The output is: 

[[1]]
[1] 1

[[2]]
[1] 1

[[3]]
[1] 1

[[4]]
[1] 1
Solution

  • Parallel computing in R works (as far as I experienced) such that for each cluster node the memory will be allocated.

    That means if you have a big data set which each node needs for calculation, this data will be allocated multiple times. This yields to high RAM consumption. Since you want to write the output in each loop and throw away the result afterwards you can try the rm function and call the garbage collection (for example with gc) in each function call.

    This worked for E L M as mention above. Thx for testing!