Search code examples
bashmakefilehpc

Why does make utility fail to delete a file associated with an incomplete task when issuing qdel command on SGE?

I have a bash script which runs a make command on a Sun Grid Engine HPC:

#submitscript.sh

module load program1 program2

set -a

make

This bash script runs a makefile which looks like:

# makefile
SHELL := /bin/bash

all: Variants.vcf Variants1.vcf

Variants.vcf:
    program1 inputfile > Variants.vcf

Variants1.vcf:
    program2 inputfile2 > Variants1.vcf

I submit the script on the HPC by issuing:

qsub submitscript.sh

When I issue qdel to delete the task the file that is currently being written to is not deleted by make. I would have thought that the qdel command would result in the makefile exiting and deleting any target files not completely written. Can anybody explain what is happening here?

Update

Thank you. I realise my mistake now. Qdel is killing the entire job so as you both say make is not being allowed to clean up. When I added the line to the makefile:

.DELETE_ON_ERROR: *.vcf

and then put a wrong file path into the program2 line eg.

Variants1.vcf:
    program2 inputfile2 > /wrong/file/path/Variants1.vcf

the desired behaviour was shown with the partially written file being deleted on exit.

Solution

  • From the Comments:

    If you kill make how can it then do the cleanup? If you killed program1 or program2 it could have a chance. – user2672165

    Response:

    I realise my mistake now. Qdel is killing the entire job so as you both say make is not being allowed to clean up. – user1977981

    I'm answering this so other users with the same problem can easily determine the answer that was provided.