I'd like to find a way to parallelize repeated independent function calls in which each call modifies the function's parent environment. Each execution of the function is independent, however, for various reasons I am unable to consider any other implementation that doesn't rely on modifying the function's parent environment. See simplified example below. Is there a way to pass a copy of the parent environment to each node? I am running this on a linux system.
create_fun <- function(){
helper <- function(x, params) {x+params}
helper2 <- function(z) {z+helper(z)}
master <- function(y, a){
parent <- parent.env(environment())
formals(parent[['helper']])$params <- a
helper2(y)}
return(master)
}
# function to be called repeatedly
master <- create_fun()
# data to be iterated over
x <- expand.grid(1:100, 1:5)
# vector where output should be stored
results <- vector("numeric", nrow(x))
# task I'd like to parallelize
for(i in 1:nrow(x)){
results[i] <- master(x[i,1], x[i, 2])
}
Functions do maintain references to their parent environments. You can look at the contents of the environment of master
(the environment created by create_fun
)
ls (environment(master) )
# [1] "helper" "helper2" "master"
Using %dopar%
you could do
## Globals
master <- create_fun()
x <- expand.grid(1:100, 1:5)
## Previous results
for(i in 1:nrow(x)){
results[i] <- master(x[i,1], x[i, 2])
}
library(parallel)
library(doParallel)
cl <- makePSOCKcluster(4)
registerDoParallel(cl)
## parallel
res <- foreach(i=1:nrow(x), .combine = c) %dopar% {
master(x[i,1], x[i,2])
}
all.equal(res, results)
# TRUE