Im trying to implemnt a multidimensional hypterparamter optimization. I have a function with many arguments but only 4 which have to be optimzized.
I wonder how i can pass this to hypteropt.
I thoughtg something like this should work in a test function:
from hyperopt import hp, fmin, tpe, Trials
def testFuntion(x,arg1='arg1',arg2='arg2',arg3='arg3'):
print(arg1,arg2,arg3)
#args1,arg2,arg3 = args
return x[0]-x[1]-x[2]+x[3]
space = ((hp.uniform('a', 0.0, 1.0),hp.uniform('b', 0.0, 1.0),hp.uniform('c', 0.0, 1.0)),hp.uniform('d', 0.0, 1.0),'blub1','blub2','blub3')
trials = Trials()
best = fmin(testFuntion, space, algo=tpe.suggest, max_evals=100)
But the function is trying to compare my strings somehow and raises Error: TypeError: unsupported operand type(s) for -: 'tuple' and 'float'
What am i doing wrong?
OK, you should take care of a few things here. The most basic one is that, if you look into your definition of space closely, the first three values (a, b and c) are actually enclosed in a 3-tuple. This is what causes your error, x[0] is that tuple and you can't subtract the float x[1] in any sensible way here.
To avoid this kind of issue, I would like to suggest that you help yourself with a better naming convention when mapping space for your function arguments. Also, the way I understand your setup, I think you should remove constants from your function signature, as this is somewhat confusing and bloats your search space unnecessarily. Here's a complete example of what you want:
from hyperopt import hp, fmin, tpe, Trials
def test_function(x):
arg1='arg1'
arg2='arg2'
arg3='arg3'
return x[0]-x[1]-x[2]+x[3]
space = [
hp.uniform('x0', 0.0, 1.0),
hp.uniform('x1', 0.0, 1.0),
hp.uniform('x2', 0.0, 1.0),
hp.uniform('x3', 0.0, 1.0)
]
trials = Trials()
best = fmin(test_function, space, algo=tpe.suggest, max_evals=100, trials=tials)
P.s.: Use the trials you defined in fmin.
Edit: although I don't quite see the point, since you don't call your optimization function directly, here's how you pass in more parameters that don't get optimized:
from hyperopt import hp, fmin, tpe, Trials
def test_function(x):
# x[4] is 'foo'
return x[0]-x[1]-x[2]+x[3]
space = [
hp.uniform('x0', 0.0, 1.0),
hp.uniform('x1', 0.0, 1.0),
hp.uniform('x2', 0.0, 1.0),
hp.uniform('x3', 0.0, 1.0),
'foo'
]
trials = Trials()
best = fmin(test_function, space, algo=tpe.suggest, max_evals=100, trials=trials)