I am just starting setup SGE (Sun Grid Engine) for cluster.
I have two Cluster Node1 and Node2,
Each Node has 24vCPU and 240GB RAM.
Problem is looks like I don't use fully memory (Around 8%) and CPU (around 30%)
Even thought I increase Thread of program looks like every time using same resource.
I want to know there is anyway to use more memory and CPU Flexible? And also what is starting point for this tunning.
I submit as below,
qsub -l mem_free=200GB -pe olcd 20 test.sh
I used qstat -f -j $jobid. and get this
parallel environment: olcd range: 20
usage 1: cpu=04:47:12, mem=30523.46624 GBs, io=0.13225, vmem=505.098M, maxvmem=3.927
That's looks like only one cpu is used eventhough I setup slot as 20.
I need to make performance better at least find best running time.
anyone has good idea to setup this Sun Grid Engine? Thanks
The slots you reserve using qsub i.e., via the -pe olcd 20 option, do not reflect what is actually used by the system. Same idea for memory, mem_free=200GB.
The only effect qsub/SGE has is to reserve these resources on behalf of the job. It is up to the job to consume these resources.
Having said all this, is the job you are submitting expected to use 20 threads and 200GB of RAM?