Search code examples
matlaboptimizationplotleast-squares

Plotting the function value evolution during all iterations using lsqnonlin

I am using lsqnonlin as my optimization routine. I need to plot the cost function at each iteration whilst showing all previous values. So I want to show something like this:

enter image description here

However, using lsqnonlin, I was only able to plot the value of the cost function at the current iteration only. using these options:

options            = optimset('TolFun', 1e-5, 'TolX',1e-5, 'MaxFunEvals', 10000, 'PlotFcns', @optimplotfval,'Display','iter')

Is there a way to set the options of the lsqnonlin such that I get something similar to the above shown figure?

Solution

  • If you look into the program for optimplotfval.m (in MATLAB's terminal enter edit optimplotfval.m you will see the following comment:

    %   STOP = OPTIMPLOTFVAL(X,OPTIMVALUES,STATE) plots OPTIMVALUES.fval.  If
%   the function value is not scalar, a bar plot of the elements at the
%   current iteration is displayed.  If the OPTIMVALUES.fval field does not
%   exist, the OPTIMVALUES.residual field is used.

    So in, for example, fminsearch you will get a plot of objective/cost function values vs. iteration count but in case of lsqnonlin it seems you are getting a bar plot of residual values at a given iteration.

    A fix to is is to make your own plotting function based on optimplotfval.m. Copy-paste optimplotfval.m into another file, e.g. my_opt_plot.m then change the residual option in the initial part of the program:

    stop = false;
switch state
    case 'iter'
        if isfield(optimValues,'fval')
            if isscalar(optimValues.fval)
                plotscalar(optimValues.iteration,optimValues.fval);
            else
                plotvector(optimValues.iteration,optimValues.fval);
            end 
        else
            % Plot the squared norm of residuals as a function of iteration number instead of bar plot of residual values at current iteration
            fval = norm(optimValues.residual)^2;
            % Call the scalar function instead
            plotscalar(optimValues.iteration,fval);    
end

    You can call this new function in the same way as you called optimplotfval.m:

    options = optimoptions('lsqnonlin','Display','iter','PlotFcns',@my_opt_plot);
[x,resnorm,residual,exitflag,output] = lsqnonlin(@simple_fun,xc0,[],[],options);

    simple_fun in my case was based on an example from MATLAB's doc entry for lsqnonlin:

    function f = simple_fun(xc)
    x = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 74.6 81.3];
    y = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 -0.4 -1.3 -1.5];

    f = xc(1)*exp(xc(2)*x)-y;   
end

    If you compare the plotted objective function values with ones printed on the screen, they indeed match.