Mean Squared Error not decresing with number of epochs?

Question

This is the implementation of Batch Gradient Descent using tensorflow.

When I am running this code the MSE remains the same.

import tensorflow as tf
from sklearn.preprocessing import StandardScaler
import numpy as np
from sklearn.datasets import fetch_california_housing

housing=fetch_california_housing()

std=StandardScaler()
scaled_housing_data=std.fit_transform(housing.data)

m,n=scaled_housing_data.shape
scaled_housing_data.shape

scaled_housing_data_with_bias=np.c_[np.ones((m,1)),scaled_housing_data]

n_epochs=1000
n_learning_rate=0.01

x=tf.constant(scaled_housing_data_with_bias,dtype=tf.float32)
y=tf.constant(housing.target.reshape(-1,1),dtype=tf.float32)
theta=tf.Variable(tf.random_uniform([n+1,1],-1.0,1.0,seed=42))
y_pred=tf.matmul(x,theta)

error=y_pred-y
mse=tf.reduce_mean(tf.square(error))
gradients=2/m*tf.matmul(tf.transpose(x),error)

training_op=tf.assign(theta,theta-n_learning_rate*gradients)

init=tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)

    for epoch in range(n_epochs):
        if epoch % 100 == 0:
            print("Epoch", epoch, "MSE =", mse.eval())
        sess.run(training_op)

    best_theta = theta.eval()

Output

('Epoch', 0, 'MSE =', 2.7544272)
('Epoch', 100, 'MSE =', 2.7544272)
('Epoch', 200, 'MSE =', 2.7544272)
('Epoch', 300, 'MSE =', 2.7544272)
('Epoch', 400, 'MSE =', 2.7544272)
('Epoch', 500, 'MSE =', 2.7544272)
('Epoch', 600, 'MSE =', 2.7544272)
('Epoch', 700, 'MSE =', 2.7544272)
('Epoch', 800, 'MSE =', 2.7544272)
('Epoch', 900, 'MSE =', 2.7544272)

The mean square error(MSE) remains the same no matter what. Please help.

Answer 1

If your MSE is same, then it means your theta is not getting updated, which implies gradients are zero. Change this line and check:

gradients=2.0/m*tf.matmul(tf.transpose(x),error) # integer division (2/m) causes zero