Gradient descent is not converging to it's minima

Question

I know this question asked many times but I still have the problem. I chose a small value of alpha and performed a good amount of iteration to find converging point but it's not working. Any help will be appreciated. Here is the full code. The GradientDescent() and Cost() function are computing the m and b values and line() and show() functions are used for only plotting the data.

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import random

def show(x,y):
    plt.plot(x,y,"ro")
    plt.show()

def line(m,b):
    xpoints=np.arange(25)
    ypoints=np.zeros(len(xpoints))

    for i in range(len(xpoints)):
        ypoints[i]=m*xpoints[i]+b

    plt.plot(xpoints,ypoints,alpha=0.2)

def cost(xpoints,ypoints,m,b,flag):
    pridicted_y=np.zeros(len(xpoints))
    error=np.zeros(len(xpoints))
    TotalError=0

    for i in range(len(xpoints)):
        if(flag==0):
            pridicted_y[i] = m*xpoints[i]+b
            error[i]= pridicted_y[i] - ypoints[i]
        if(flag==1):
            pridicted_y[i] = m*xpoints[i]+b
            error[i]= (pridicted_y[i] - ypoints[i])*xpoints[i]
        TotalError=TotalError+error[i]

        # plt.plot([xpoints[i],xpoints[i]],[ypoints[i],pridicted_y[i]])
        # print(error[i],end=" ")
    return TotalError
def GradientDescent(xpoints,ypoints,m,b,alpha):
    k=len(xpoints)
    M=m
    B=b
    x=0
    for i in range(500):
        for j in range(2):
            M = m-alpha*(1/k)*cost(xpoints,ypoints,m,b,0)
            B = b-alpha*(1/k)*cost(xpoints,ypoints,m,b,1)
        m=M
        b=B
        line(m,b)
        if(x==1):
            plt.show()
            print(i,m,b)
    return (m,b)
#random data set
x=np.arange(20)
y = [int(i+random.random()*10) for i in x]
min_x = np.mean(x)
min_y = np.mean(y)


#predetermined variable values
m=0
b=0
alpha=0.001

# cost(x,y,m,b)
m,b=GradientDescent(x,y,m,b,alpha)
plt.plot(min_x,min_y,"ko")
plt.plot(min_x,m*min_x+b,"go")
line(m,b)
show(x,y)
print(m,b)

Answer 1

It seems that you have just made a small mistake: in your cost function which computes the gradient, the gradient w.r.t m has been replaced with the gradient w.r.t b and vice versa. By changing the flags as follows

    if(flag==1):#instead of flag==0
        pridicted_y[i] = m*xpoints[i]+b
        error[i]= pridicted_y[i] - ypoints[i]
    if(flag==0):#instead of flag==1
        pridicted_y[i] = m*xpoints[i]+b
        error[i]= (pridicted_y[i] - ypoints[i])*xpoints[i]

I get this result: