Search code examples
rparallel-processingplyrmulticoreargument-passing

How do I set preschedule=FALSE for llply's parallel mode?


I want to use llply to perform some parallel computations using the multicore doParallel backend (i.e. doParallel::registerDoParallel(cores=8)), but each computation will take a different amount of time, so I want to set the multicore preschedule option to false for this computation. However, I'm unsure how to pass this option to llply which will pass it to foreach which will pass it to doParallel which will finally pass it to mclapply. Can anyone give an example of doing this?

In other words, how would I disable prescheduling in the following code?

library(plyr)
library(doParallel)
registerDoParallel(cores=2)
x <- llply(1:10, sqrt, .parallel=TRUE)

Solution

  • You can use the llply ".paropts" option to do that:

    opts <- list(.options.multicore=list(preschedule=FALSE))
    x <- llply(1:10, sqrt, .parallel=TRUE, .paropts=opts)
    

    As you guessed, the ".paropts" option passes the ".options.multicore" option to foreach, which propagates it on to doParallel, which calls mclapply with mc.preschedule=FALSE.