Search code examples
pythonmachine-learninggradient-descent

I am unable to get this gradient descent solution correct


Consider a linear-regression model with N=3 and D=1 with input-output pairs as follows:

yl=22, x 1=1, y2=3, x2=1, y3=3, x3=2

What is the gradient of mean-square error (MSE) with respect to B1 (when Bo=0 and B1=1? Give your answer correct to two decimal digits.


Solution

  • MSE Loss = sum((h - y) ** 2) / 2m
    
    Gradient wrt b1 will be sum[(h - y) . x)] / m:
    
    hypothesis: h = b0 + b1.x
    
    for b0 = 0, b1 = 1:
    
    h = x
    
    input(x)        : [  1,  1,  2]
    
    prediction(h)   : [  1,  1,  2]
    
    Ground truth(y) : [ 22,  3,  3]
    
    h - y           : [-21, -2, -1]
    
    (h - y). x      : [-21, -2, -2]
    
    gradient(b1)    : (-21 - 2 - 2) / 3 = -25 / 3 = -8.3333