I am using python package hyperopt and I have a parameter a which requires to be larger than parameter b.
For example, I hope my parameter space is like
from hyperopt import hp
space = {"b": hp.uniform(0, 0.5), "a": hp.uniform(b, 0.5)}
Which, requires a to be at least larger than b, how can I do that?
Thanks in advance
A simple option is to use the ability of hyperopt to nest parameters. You can thus define a hyper-parameter space like you want:
space = hp.uniform("a", hp.uniform("b", 0, 0.5), 0.5)
Only "a"'s value is passed to the function that you optimize (because this is the hyper-parameter space), but hyperopt.fmin() will return both parameters.
A similar option, but where the function to be optimized receives both parameters is:
b_var = hp.uniform("b", 0, 0.5)
space = {"b": b_var, "a": hp.uniform("a", b_var, 0.5)}
Finally, it might be simpler to change a bit the inputs to the optimized function: parameter a can be replaced by a_fraction running between 0 and 1 and interpolating between b and 0.5 (i.e. a_fraction = 0 yields a = b and a_fraction = 1 gives a = 0.5 inside the modified function to be optimized). The parameter space thus has the usual form:
space = {"b": hp.uniform("b", 0, 0.5), "a_fraction": hp.uniform("a_fraction", 0, 1)}
There is an interesting discussion at https://github.com/hyperopt/hyperopt/issues/175#issuecomment-29401501.