Search code examples
javamultithreadingparallel-processingsungridengine

SGE : Parallel Environment for a multithreaded java code


I have written a multi threaded java code, which when runs creates 8 threads and the computation continues on these threads. I would like to submit this job to a SGE cluster but I am not sure which parallel environment (pe) should I choose? or should I create one? I am new to SGE. The simple way would be to run it in serial mode, but that's inefficient.

Regarding creating a pe, where does it need to be created? Should the SGE deamon also need to have this pe ? When I submitted a job with some random name as pe, I got

 job rejected: the requested parallel environment "openmpi" does not exist

Solution

  • Threaded applications must get all their slots on a single node. That's why you need a Parallel Environment with allocation_rule set to $pe_slots. Parallel environments are configured by the SGE administrator using the qconf -ap PE_name. As a user you can only get the list of the available PEs using qconf -spl and can query the configuration of a particular PE using qconf -sp PE_name. You can walk all PEs and see their allocation rules with the following (ba)sh script:

    for pe_name in `qconf -spl`; do
      echo $pe_name
      qconf -sp $pe_name | grep allocation_rule
    done
    

    But you should already be talking to your SGE admin instead of trying to justify your off-topic question here.