Dynamically choose variables to be minimized in multidimensional optimization

Question

In Matlab, I know how to program a multidimensional optimization. But I would like to dynamically choose a subset of variables to be optimized.

Suppose I have a three-dimensional variable vector, but I want Matlab to only optimize the first and the second variable. How can this be achieved?

x1 = 0:5; p_true = [1 2 3];             % True parameters
y1 = polyval(p_true, x1);               % True data
yn = y1 + sin(x1);                      % Noisy data

optimizationOptions = optimset('Display', 'final', 'TolFun', 1e-7, 'MaxFunEvals', 1e5,...
    'MaxIter', 1e4);
p0 = [0.5 0.75 1];                      % Initial guess
[p1, ~, ~, optOut] = fminsearch(@(parameters) objFunB(parameters, x1, yn), p0,...
    optimizationOptions);

plot(x1, yn, 'rx');
hold on
plot(x1, polyval([p1(1:2) 3], x1), 'b');

function rmse = objFunB(parameters, x1, yn)
    % Manipulate third component to be fixed; still, fminsearch tries adjusting it
    parameters(3) = 3;
    rmse = sum((yn - polyval(parameters, x1)).^2);
end

This clumsy solution tricks fminsearch into considering the third variable as insensitive, since it is overwritten inside of the objective function and thus, does not affect the output value.

Defining the third value as a separate variable (i.e., outside parameters) is not an option, since this would require considerable recoding each time I choose a different variable to be optimized.

There must be a better solution. Any ideas?

Answer 1

While bounded optimization with lb == ub is possible, it limits the available algorithms to those which accept constraints (and may also have a performance impact, which I have yet to test). The built-in fminsearch does not facilitate bounded multidimensional optimization (although fminsearchbnd does).

I have come up with the following solution, starting from line 4 of the original script. It makes use of logical indexing.

all_parameters = p_true';
logOfPar2Opt = [1 0 1]';
p0 = nonzeros(all_parameters.*logOfPar2Opt); % Initial guess
optimizationOptions = optimset('Display', 'final', 'TolFun', 1e-7, 'MaxFunEvals', 1e5,...
    'MaxIter', 1e4);
[p1, fval, ~, optInfo] = fminsearch(@(parameters) objFunB(parameters, logOfPar2Opt,...
        all_parameters, x1, yn), p0, optimizationOptions);
indOfPar2Opt = find(logOfPar2Opt);
p_opt = all_parameters;
p_opt(indOfPar2Opt) = p1;

plot(x1, yn, 'rx');
hold on
plot(x1, polyval(p_opt, x1), 'b');

%% Separate objective functions
function rmse = objFunB(par2opt, logOfPar2Opt, all_parameters, x1, yn)
    indOfPar2Opt = find(logOfPar2Opt);
    prms = all_parameters;
    prms(indOfPar2Opt) = par2opt;
    rmse = sum((yn - polyval(prms, x1)).^2);
end