How to access standard outputs while the process is running on a computer cluster?

Question

Intro

I use a computer cluster on which I submit (qsub) jobs via a .pbs file. The default is to print standard outputs and standard errors on files in the working directory from which the qsub command is executed.

Issue

The issue for me is that the standard output and standard error files appear only at the end of the process while I would love to be able to monitor the advances of the process in direct.

Thoughts

I know the following options exist

#PBS -o
#PBS -e
#PBS -j [oe/eo]

in order to indicate specific file names and locations for standard outputs and standard errors. However, two points are unclear to me...

1. I don't quite know how it works when using array (via #PBS -t). Will standard outputs from one PBS_ARRAYID overwrite previous standard outputs? Do all the standard outputs for the whole array get printed on the same file or is there a way to include the PBS_ARRAYID in the file name?

2. Would using these options help me in any way to access the standard outputs before the end of the process?

Question

How can I visualize the standard outputs and standard errors before a process is over when the process is submitted on a computer cluster?

Answer 1

In the computer cluster I am using at least the standard outputs are kept in the RAM saved to the #PBS -o at the end of the process only. One way around that is to explicitly redirect the standard outputs in the shell script. Do something like:

executable arguments >& StandardOutputs_${PBS_JOBNAME}.txt