Gradient descent implementation is not working in Julia

Question

I am trying to Implement gradient Descent algorithm from scratch to find the slope and intercept value for my linear fit line.

Using the package and calculating slope and intercept, I get slope = 0.04 and intercept = 7.2 but when I use my gradient descent algorithm for the same problem, I get slope and intercept both values = (-infinity,-infinity)

Here is my code

x= [1,2,3,4,5,6,7,8,9,10,11,12,13,141,5,16,17,18,19,20]
y=[2,3,4,5,6,7,8,9,10,11,12,13,141,5,16,17,18,19,20,21]    

function GradientDescent()
   m=0
   c=0
   for i=1:10000
      for k=1:length(x)
         Yp = m*x[k] + c   
         E = y[k]-Yp             #error in predicted value

         dm = 2*E*(-x[k])    # partial derivation of cost function w.r.t slope(m)
         dc = 2*E*(-1)       # partial derivate of cost function w.r.t. Intercept(c)
         m = m + (dm * 0.001)
         c = c + (dc * 0.001)
      end
   end
   return m,c
end

Values  = GradientDescent()   # after running values = (-inf,-inf)

Answer 1

I have not done the math, but instead wrote the tests. It seems you got a sign error when assigning m and c.

Also, writing the tests really helps, and Julia makes it simple :)

function GradientDescent(x, y)
   m=0.0
   c=0.0
   for i=1:10000
      for k=1:length(x)                       
        Yp = m*x[k] + c   
        E = y[k]-Yp
        dm = 2*E*(-x[k])
        dc = 2*E*(-1)

        m = m - (dm * 0.001)
        c = c - (dc * 0.001)
       end
   end
   return m,c
end

using Base.Test

@testset "gradient descent" begin
    @testset "slope $slope" for slope in [0, 1, 2]
        @testset "intercept for $intercept" for intercept in [0, 1, 2]
            x = 1:20
            y = broadcast(x -> slope * x + intercept, x)

            computed_slope, computed_intercept = GradientDescent(x, y)
            @test slope ≈ computed_slope atol=1e-8
            @test intercept ≈ computed_intercept atol=1e-8
        end
    end
end