Search code examples
rparallel-processingr-future

What is the idiomatic, flexible way to specify mutliprocess plan in future?

I just encountered the R packages furrr and future. I would love to use them to write flexible code that will use multiple cores if available on machines running Windows or OSX. I would love for the default # of "available" cores to be something like parallel::detectCores()-1, rather than detectCores(). Seems like plan(multiprocess) is the streamlined, idiomatic way to almost do this, but it defaults to using all the cores on the machine. I've encountered more explicit ways to specify the "plan". What is the idiomatic way to use mostly the default behavior of plan(multiprocess), but also limit the number of cores to 1 fewer than whatever detectCores() would return?

EDIT Based on @HernikB 's comment above, I believe that a reasonable answer to this question would be along the lines of options(future.plan="multiprocess",mc.cores = parallel::detectCores() - 1L).

EDIT I am finding that this method frequently doesn't use the multiprocess plan (or at least does not go parallel with future_map_dfr) when it seems it could. By contrast

nc<-detectCores()-1 plan(strategy=multiprocess, workers=nc)

seems to get it going in parallel. Leaving the question as unanswered.

Solution

  • Take a look at ?future::multiprocess. You can do:

    plan(multiprocess(workers = 3))

    If you have 4 cores on your compute.r.