Search code examples
command-linecommandjobslsf

Best practice for submitting long commands to LSF

I'm submitting jobs to a cluster using bsub in LSF. My command calls third party software and contains a long list of files, along with other arguments (around 8000 characters).

Is it safe to submit such long commands directly or will they get truncated? Should I be saving the command to file then calling bsub sh command.sh?

What is the best practice for submitting such long commands.

Thanks

Solution

  • There's no special limitation in LSF (that I know of) that will truncate an entire command line outright. There are a few things you might want to consider, however:

    1. The shell you run the bsub command in might impose a limit on the total length of the command you can submit. This is often controlled by an OS limit, there are many questions about it floating around the internet. Here is one example which shows a couple of ways to determine the limit on your system. In case of broken links, the top answer there recommends xargs --show-limits while another way is to check getconf ARG_MAX
    2. Certain bsub options that accept strings as inputs may have limits on the length of the string that can be passed to them. For example the -R option can accept strings of 512 characters -- for the most part I believe LSF will reject the job if the argument is too long (but I can't guarantee for sure if that's the case for all of them).