Search code examples
rparallel-processingcran

Doesn't win-builder support parallelisation in vignettes or is this just bad practice?


I'm creating a package with some embarrassingly parallel computations and am using the doParallel package to parallelise these. I moved to doParallel as doMC doesn't work on Windows. Since these are core functions in my package, the vignettes also establish a parallel backend.

R CMD check passes without any warnings on my Linux distro and on my colleague's Windows machine, however when submitting it to win-builder (see http://win-builder.r-project.org/), for carrying out an R CMD check on a Windows server I get the following error when it checks the vignettes:

* checking running R code from vignettes ... [15s] ERROR
Errors in running code in vignettes:
when running code in 'my_vignette.Rmd'

When sourcing 'my_vignette.R':
Error: 16 simultaneous processes spawned
Execution halted

As you can see, in the function I use half of the machine cores for the computation (as in the vignette this cannot be set to a user-specific variable). So my question is -- is this a problem with win-builder or is it just considered bad practice to have a parallel backend in a vignette? I should say that the package has functions which are quite computationally intensive so the vignettes will also be quite computationally demanding.

Thank you for any comments.


Solution

  • I was told my the maintainer of the win-builder that for resource management, the number of processes on the parallel backend is limited to TWO. This is also a CRAN policy. Thanks to Uwe for the prompt reply.