How do I make Hydra's multirun sweeper test multiple repetitions of each condition?

Question

Hydra enables sweeping combinations of parameters, for example:

python my_app.py --multirun db=mysql,postgresql schema=warehouse,support,school

will test six different conditions.

Is there a principled way of conducting multiple runs of each condition (other than adding a dummy factor)?

Answer 1

The typical purpose of doing something like this is to run non-deterministic functions. The canonical way is to add a random seed parameter and to use it to initialize the random subsystem(s).

Note that you can use range to make this shorter, something like:

$ python my_app.py --multirun a=a1,a2 b=b1,b2,b3 'seed=range(1,5)'

If you want more sophisticated handling of the sweep, take a look at the Sweeper plugins (e.g the Optuna sweeper).