Search code examples
bashzshgnu-parallel

Enabling/disabling binary flags in GNU Parallel

I am trying to use GNU Parallel to run a script that has multiple binary flags. I would like to enable/disable these as follows:

Given a script named "sample.py", with two options, "--seed" which takes an integer and "--something" which is a binary flag and takes no input, I would like to construct a call to parallel that produces the following calls:

python sample.py --seed 1111
python sample.py --seed 1111 --something
python sample.py --seed 2222
python sample.py --seed 2222 --something
python sample.py --seed 3333
python sample.py --seed 3333 --something

I've tried things like

parallel python sample.py --seed {1} {2} ::: 1111 2222 3333 ::: "" --something
parallel python sample.py --seed {1} {2} ::: 1111 2222 3333 ::: '' --something
parallel python sample.py --seed {1} {2} ::: 1111 2222 3333 ::: \  --something

but haven't had any luck. Is what I'm trying to achieve possible with GNU parallel? I can modify my script to take explicit TRUE/FALSE values for the flag but I'd prefer to avoid that if possible.

Solution

  • You are so close.

    GNU Parallel quotes replacement strings. That usually makes sense, because it is then safe to give it filenames like:

    My brother's 12" records, all with ***.csv

    which could otherwise give no end of troubles.

    However, to be consistent GNU Parallel also quotes the empty string. And that is what is hitting you here.

    --dry-run shows what is going on:

    $ parallel --dry-run python sample.py --seed {1} {2} ::: 1111 2222 3333 ::: '' --something
python sample.py --seed 1111 ''
python sample.py --seed 1111 --something
python sample.py --seed 2222 ''
python sample.py --seed 2222 --something
python sample.py --seed 3333 ''
python sample.py --seed 3333 --something

    So how can you avoid that?

    You can tell the shell to evaluate all strings:

    parallel eval python sample.py --seed {1} {2} ::: 1111 2222 3333 ::: '' --something

    but that might be a bit of a blunt hammer when you need a scalpel. From version 20190722 you can also use {=uq=}. uq() is a perl function which tells GNU Parallel that this replacement string should not be quoted:

    $ parallel-20190722 --dry-run python sample.py --seed {1} {=2 uq=} ::: 1111 2222 3333 ::: '' --something
python sample.py --seed 1111 
python sample.py --seed 1111 --something
python sample.py --seed 2222 
python sample.py --seed 2222 --something
python sample.py --seed 3333 
python sample.py --seed 3333 --something