Search code examples
pythonscipy-optimize

Scipy least squares: is it possible to optimize for two error functions simultaneously?


For this particular work, I am using scipy optimize to try find the best parameters that fit two different models at the same time.

model_func_par = lambda t, total, r0, theta:  np.multiply((total/3),(1+2*r0),np.exp(-t/theta))
model_func_perp = lambda t, total, r0, theta:  np.multiply((total/3),(1-r0),np.exp(-t/theta)) 

After this I create two error functions by subtractig the raw data, and plug it into scipy.optimize.leastsq(). As you can see I have two different equations with the same r0 and theta parameters - I have to find the parameters that fit best both equations (in theory, r0 and theta should be the same for both equations, but because of noise and experimental errors etc I am sure this won't be quite the case.

I guess I could do a separate optimization for each equation and perhaps take an average of the two results, but I wanted to see if anyone knows of a way to do one optimization for both.

Thanks in advance!


Solution

  • Is there any specific reason to use np.multiply? Since the typical mathematical operators are overloaded for np.ndarrays, it's more convenient to write (and read):

    model1 = lambda t, total, r0, theta: (total/3) * (1+2*r0) * np.exp(-t/theta) 
    model2 = lambda t, total, r0, theta: (total/3) * (1-r0) * np.exp(-t/theta) 
    

    To answer your question: AFAIK this isn't possible with scipy.optimize.least_squares. However, a very simple approach would be to minimize the sum of least squares residuals

    min || model1(xdata, *coeffs) - ydata ||^2 + || model2(xdata, *coeffs) - ydata ||^2
    

    like this:

    import numpy as np
    from scipy.optimize import minimize
    from scipy.linalg import norm
    
    # your xdata and ydata as np.ndarrays
    # xdata = np.array([...])
    # ydata = np.array([...])
    
    # the objective function to minimize
    def obj(coeffs):
        return norm(model1(xdata,*coeffs)-ydata)**2 + norm(model2(xdata,*coeffs)-ydata)**2
    
    # the initial point (coefficients)
    coeffs0 = np.ones(3)
    
    # res.x contains your coefficients
    res = minimize(obj, x0=coeffs0)  
    

    However, note that this isn't a complete answer. A better approach would be to formulate it as a multi-objective optimization problem. You may take a look at pymoo for this purpose.