Search code examples
nextflow

why I cant assign specific cpu number to a process in nextflow

I defined it in nextflow.config like:

process.executor = 'sge'
process.clusterOptions = { "-clear -cwd -l vf=20g,num_proc=24  -binding linear:24 -P W21P0024 -q mgi.q"}

and in a separate process,

process A {
  cpus 1
  memory '1 GB'
  
  input:
  etc...
}

but it still use 24 cpus like the setting in the config file. how to correctly set this?

Solution

  • When the sge executor tries to return a list of directives to use, the clusterOptions, if any, will be applied last1. I suspect these override existing ones, including -l slots=1 which is what your process A will try to use to ask for one cpu. I think in the first instance, perhaps try using a more simplified configuration. Note that the clusterOptions directive does not need to be a closure:

    process {
    executor = 'sge'
    clusterOptions = "-P W21P0024"
    queue = "mgi.q"
}