I want to submit several "cat jobs" on the fly to a cluster using qsub. Currently I'm concatenating several files with cat to a single one at the end (using > output_file) at the end of the command.
The problem is that qsub takes the > output_file from the command as part of the qsub, putting there the log of the job instead of the cat output.
qsub -b y -cwd -q bigmem cmd1
where cmd1 looks like:
cat file1 file2 filen > output_file
When a job is running via pbs, stdout is redirected to the output file of the job, so the way to do this is to write a script:
#!/bin/bash
cat file1 file2 ... filen
You don't need to redirect the output to a file because the mom daemon will do that for you in setting up the job, you just need to specify the output file you desire using -o. For example, if you named the above script script.sh (make sure it is executable) you'd submit:
qsub script.sh -b y -q bigmem -o output_file