I am trying to implement/solve the first programming excersise from Andrew ng`s machine learn cours on coursera. I have trouble implementing linear gradient descent (for one variable) in octave. I don't get the same paramters values back like in the solution but my parameters goes in the same direction (at least I think so). So I may have somewhere in my code a bug. Maybe someone who has more experience than me can enlighten me.
function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta
% theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by
% taking num_iters gradient steps with learning rate alpha
% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);
theta1 = theta(1);
theta2 = theta(2);
temp0 = 0;
temp1 = 0;
h = X * theta;
for iter = 1:(num_iters)
% ====================== YOUR CODE HERE ======================
% Instructions: Perform a single gradient step on the parameter vector
% theta.
%
% Hint: While debugging, it can be useful to print out the values
% of the cost function (computeCost) and gradient here.
%
temp0 = 0;
temp1 = 0;
for i=1:m
error = (h(i) - y(i));
temp0 = temp0 + error * X(i, 1));;
temp1 = temp1 + error * X(i, 2));
end
theta1 = theta1 - ((alpha/m) * temp0);
theta2 = theta2 - ((alpha/m) * temp1);
theta = [theta1;theta2];
% ============================================================
% Save the cost J in every iteration
J_history(iter) = computeCost(X, y, theta);
end
end
My exspected results for excersise 1 with theta initialized with [0;0] should be for theta1: -3.6303 and for theta2: 1.1664
But I become as output theta1 is 0.095420 and thetha2 is 0.51890
This is the formula I use for linear gradient descent.
EDIT1: Edited code. Now I got for theta1:
87.587
And for theta2
979.93
I now know what my problem was. I am going to describe it quick for anbody who might be intrested in it. So i accidently calulated the avriable h
outside of my loop. So every time in the loop it calulated with the same value.
Here is the fixed code:
function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta
% theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by
% taking num_iters gradient steps with learning rate alpha
% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);
theta1 = theta(1);
theta2 = theta(2);
temp0 = 0;
temp1 = 0;
error = 0;
for iter = 1:(num_iters)
% ====================== YOUR CODE HERE ======================
% Instructions: Perform a single gradient step on the parameter vector
% theta.
%
% Hint: While debugging, it can be useful to print out the values
% of the cost function (computeCost) and gradient here.
%
h = X * theta; %heres the variable i moved into the loop
temp0 = 0;
temp1 = 0;
for i=1:m
error = (h(i) - y(i));
temp0 = temp0 + (error * X(i, 1));
temp1 = temp1 + (error * X(i, 2));
%disp(error);
end
theta1 = theta1 - ((alpha/m) * temp0);
theta2 = theta2 - ((alpha/m) * temp1);
theta = [theta1;theta2];
% ============================================================
% Save the cost J in every iteration
J_history(iter) = computeCost(X, y, theta);
end
end