I'm very new to optimization algorithms. I'm trying to do a particle swarm optimization in python with pyswarms
, but I'm obviously missing something as I'm constantly getting errors like:
ValueError: operands could not be broadcast together with shapes (1,40,10) (10,40) (10,40)
There are 40 variables in the function, and here I've chosen 10 particles. I don't understand where the first three-dimensional array is coming from. For the other two they seem to be the input array copied for each particle.
Here is the code of the objective function:
def svm_func(x_in, model, target_disp=0, npart=10):
feats = tuple(model.feature_names_in_)
nfeat = model.n_features_in_
nsupp = int(model.n_support_)
support_vectors = model.support_vectors_
dual_coefs = model.dual_coef_[0]
fx = (np.matmul(dual_coefs.reshape(1,nsupp),
rbf_kernel(support_vectors.reshape(nsupp,nfeat),
Y=x_in.reshape(npart,nfeat),
gamma=model.get_params()['gamma']))
+ model.intercept_)
fx_err = (fx - target_disp)**2
return fx_err
And for the optimization:
feats = tuple(model.feature_names_in_)
x_opt_vars = {'E1':[-3,3],'E5':[-3,3]} # optimizing only these 2 out of the 40 vars
bounds = (np.array([arr_input[feats.index(i)] if i not in x_opt_vars.keys() else x_opt_vars[i][0] for i in feats]),
np.array([arr_input[feats.index(i)] if i not in x_opt_vars.keys() else x_opt_vars[i][1] for i in feats]))
options = {'c1': 0.5, 'c2': 0.3, 'w': 0.9}
optimizer = GlobalBestPSO(n_particles=10, dimensions=40, options=options, bounds=bounds)
cost, pos = optimizer.optimize(svm_func, iters=1000, model=ksvr, target_disp=-2.78, npart=10)
I thought I messed up somewhere in the "complicated" function, so I tried to create a simple parabola and find the minimum but the same occurs. The parabola:
def test_fun(x):
fx = x ** 2
return fx
options = {'c1': 0.5, 'c2': 0.3, 'w': 0.9}
optimizer = GlobalBestPSO(n_particles=1, dimensions=1, options=options)
optimized = optimizer.optimize(test_fun, iters=1000)
And here I'm getting a similar error:
ValueError: non-broadcastable output operand with shape (1,1) doesn't match the broadcast shape (1,1,1)
The problem was in the lower and upper bounds set to the same value for the features which were held constant. To solve the issue, a small constant of 1e-10 was subtracted from the values to form the lower bounds, and added to form the upper bounds. After that the optimization run just fine.