Pool performance for a simple linear regression model trained by TensorFlow2

Question

My model as simple as y = 2*x + 200 + error, but I cannot get proper result in a simple way. I don't know what happened.

import numpy as np
from tensorflow import keras
x = np.arange(100)
error = np.random.rand(100,1).ravel()
y = 2*x + 200 + error

opt = keras.optimizers.Adam(lr=0.0005)
model = keras.Sequential([keras.layers.Dense(1, input_shape=[1])])
model.compile(optimizer=opt, loss='mse', metrics=['mae'])
early_stopping_callback = keras.callbacks.EarlyStopping(
        patience=10,
        monitor='val_loss',
        mode='min',
        restore_best_weights=True)
history = model.fit(x, y, epochs=2000, batch_size=16, verbose=1,
                    validation_split=0.2, callbacks=[early_stopping_callback])

My model always stoped when validation loss was big:

Epoch 901/2000 5/5 [==============================] - 0s 3ms/step - loss: 14767.1357 - val_loss: 166.8979

And I keep getting improper after-trained parameters:

model.weights

[<tf.Variable 'dense_28/kernel:0' shape=(1, 1) dtype=float32, numpy=array([[4.2019334]], dtype=float32)>,
<tf.Variable 'dense_28/bias:0' shape=(1,) dtype=float32, numpy=array([2.611792], dtype=float32)>]

Please help me to figure out what's wrong with my code.

I use tensorflow-v2.3.0

Answer 1

I got it, the main problem is that EarlyStopping stopped my training process too early! Another problem is learning rate is too small.

So when I changed two paremeter setting, I got proper result:

import numpy as np
from tensorflow import keras
x = np.arange(100)
error = np.random.rand(100,1).ravel()
y = 2*x + 200 + error

opt = keras.optimizers.Adam(lr=0.8)  # <--- bigger lr
model = keras.Sequential([keras.layers.Dense(1, input_shape=[1])])
model.compile(optimizer=opt, loss='mse', metrics=['mae'])
early_stopping_callback = keras.callbacks.EarlyStopping(
        patience=100,  # <--- longer patience to training
        monitor='val_loss',
        mode='min',
        restore_best_weights=True)
history = model.fit(x, y, epochs=2000, batch_size=16, verbose=1,
                    validation_split=0.2, callbacks=[early_stopping_callback])