Search code examples
pythonoptimizationparticle-swarm

How are arrays formed in particle swarm optimization with pyswarms?


I'm very new to optimization algorithms. I'm trying to do a particle swarm optimization in python with pyswarms, but I'm obviously missing something as I'm constantly getting errors like:
ValueError: operands could not be broadcast together with shapes (1,40,10) (10,40) (10,40)
There are 40 variables in the function, and here I've chosen 10 particles. I don't understand where the first three-dimensional array is coming from. For the other two they seem to be the input array copied for each particle. Here is the code of the objective function:

def svm_func(x_in, model, target_disp=0, npart=10):
    feats = tuple(model.feature_names_in_)   
    nfeat = model.n_features_in_
    nsupp = int(model.n_support_)
    support_vectors = model.support_vectors_
    dual_coefs = model.dual_coef_[0]

    fx = (np.matmul(dual_coefs.reshape(1,nsupp), 
                    rbf_kernel(support_vectors.reshape(nsupp,nfeat), 
                               Y=x_in.reshape(npart,nfeat), 
                               gamma=model.get_params()['gamma']))
          + model.intercept_)
    
    fx_err = (fx - target_disp)**2
    return fx_err 

And for the optimization:

feats = tuple(model.feature_names_in_)
x_opt_vars = {'E1':[-3,3],'E5':[-3,3]} # optimizing only these 2 out of the 40 vars
bounds = (np.array([arr_input[feats.index(i)] if i not in x_opt_vars.keys() else x_opt_vars[i][0] for i in feats]), 
          np.array([arr_input[feats.index(i)] if i not in x_opt_vars.keys() else x_opt_vars[i][1] for i in feats]))
options = {'c1': 0.5, 'c2': 0.3, 'w': 0.9}
optimizer = GlobalBestPSO(n_particles=10, dimensions=40, options=options, bounds=bounds)

cost, pos = optimizer.optimize(svm_func, iters=1000, model=ksvr, target_disp=-2.78, npart=10)

I thought I messed up somewhere in the "complicated" function, so I tried to create a simple parabola and find the minimum but the same occurs. The parabola:

def test_fun(x):
    fx = x ** 2
    return fx

options = {'c1': 0.5, 'c2': 0.3, 'w': 0.9}
optimizer = GlobalBestPSO(n_particles=1, dimensions=1, options=options)

optimized = optimizer.optimize(test_fun, iters=1000)

And here I'm getting a similar error: ValueError: non-broadcastable output operand with shape (1,1) doesn't match the broadcast shape (1,1,1)


Solution

  • The problem was in the lower and upper bounds set to the same value for the features which were held constant. To solve the issue, a small constant of 1e-10 was subtracted from the values to form the lower bounds, and added to form the upper bounds. After that the optimization run just fine.