Search code examples
rlinuxparallel-processingr-future

Kill futures from future_apply() on Linux

I use future_lapply() to parallel my code on a Linux machine. If I terminate the process early, only one worker is freed and the parallel processes continue to persist. I know I can enter tools::pskill(PID) to end each individual process, but this is tedious as I run on 26 cores.

If there a way to make a system call to linux, from R, to get all the active PIDs?

I set up future_lapply as such:

# set number of workers
works <- 26
plan(multiprocess, workers = works)
future_lapply(datas, function(data) {
  # do some long processes
}

If I terminate the process and run top I will still see: enter image description here

As my parallel sessions are still running.

Update with session information:
version.string R version 3.6.2 (2019-12-12)
future 1.12.0
future.apply 1.2.0

Solution

  • I hope this helps. 

    require(future)
works <- 26
plan(multiprocess, workers = works)
future_lapply(datas, function(data) {
  # do some long processes
})

# get all PIDs of the r processess
v <- listenv::listenv()  # requires listenv package
for (ii in 1:works) {
   v[[ii]] %<-% {
         Sys.getpid()
     }
}

for (i in 1:works) {
  #For windows
  system(sprintf("taskkill /F /PID %s", v[[i]]))

  #For Linux
  system(sprintf("kill -9 %s", v[[i]]))
}

    Have a great day.