I'm taking Andrew Ng's Machine Learning course on coursera, and I'm pretty confused as to why one particular example works in the One vs. All classification:
function [all_theta] = oneVsAll(X, y, num_labels, lambda)
%ONEVSALL trains multiple logistic regression classifiers and returns all
%the classifiers in a matrix all_theta, where the i-th row of all_theta
%corresponds to the classifier for label i
% [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels
% logisitc regression classifiers and returns each of these classifiers
% in a matrix all_theta, where the i-th row of all_theta corresponds
% to the classifier for label i
% Some useful variables
m = size(X, 1);
n = size(X, 2);
% You need to return the following variables correctly
all_theta = zeros(num_labels, n + 1);
% Add ones to the X data matrix
X = [ones(m, 1) X];
% ====================== YOUR CODE HERE ======================
% Instructions: You should complete the following code to train num_labels
% logistic regression classifiers with regularization
% parameter lambda.
%
% Hint: theta(:) will return a column vector.
%
% Hint: You can use y == c to obtain a vector of 1's and 0's that tell use
% whether the ground truth is true/false for this class.
%
% Note: For this assignment, we recommend using fmincg to optimize the cost
% function. It is okay to use a for-loop (for c = 1:num_labels) to
% loop over the different classes.
%
% fmincg works similarly to fminunc, but is more efficient when we
% are dealing with large number of parameters.
%
% Example Code for fmincg:
%
% % Set Initial theta
% initial_theta = zeros(n + 1, 1);
%
% % Set options for fminunc
% options = optimset('GradObj', 'on', 'MaxIter', 50);
%
% % Run fmincg to obtain the optimal theta
% % This function will return theta and the cost
% [theta] = ...
% fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ...
% initial_theta, options);
%
initial_theta = zeros(n + 1, 1);
options = optimset('GradObj', 'on', 'MaxIter', 50);
for i = 1:num_labels
c = i * ones(size(y));
fprintf('valores')
[theta] = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options);
all_theta(i,:) = theta;
end
% =========================================================================
end
I'm particularlly confused about the line: [theta] = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options);. lrCostFunction is defined to have parameters theta, X, y, lambda, so I don't know what the t is doing there.
Also, what is the purpose of wrapping theta in brackets: [theta]?
Any help on stepping through this code would be great. Thanks.
You are looking at a line where an anonymous function is defined. An anonymous function is like a short-hand definition of a function, and starts with an @, followed by parameters of this function (in your case t). This parameter t is passed on to the function lrCostFunction() as the first parameter, and is in fact the theta parameter. I.e., you are asking the function fmincg() to minimize the output of this anonymous function, which is a wrapper around lrCostFunction() so that you minimize over theta, while using X, y and lambda as defined outside the anonymous function definition.
To get a better understanding of anonymous functions, you can split the code:
func_handle = @(t)(lrCostFunction(t, X, (y == c), lambda)) % anonymous function
func_handle(initial_theta); % returns the cost at the initial_theta
[theta] = fmincg(func_handle, initial_theta, options);
See the official Matlab documentation for details on anonymous functions.
The brackets around theta are redundant.