For a multiple linear regression model in Tensorflow in python, how can you print out the equation that the model is using to predict the label. The model I am currently using takes two features to predict one label, so I think the general equation is this but how could I get the unknown parameters and values of all the constants using Tensorflow?
Code:
fundingFeatures = fundingTrainSet.copy()
fundingLabels = fundingFeatures.pop('% of total funding spent')
fundingFeatures = np.array(fundingFeatures)
normalizer = preprocessing.Normalization()
normalizer.adapt(fundingFeatures)
model = tf.keras.Sequential([
normalizer,
layers.Dense(units=1)
])
model.compile(loss = tf.losses.MeanSquaredError(),
optimizer = tf.keras.optimizers.SGD(
learning_rate=0.06, momentum=0.0, nesterov=True, name="SGD",
))
model.fit(fundingFeatures, fundingLabels, epochs=1000)
I will explain how you can write the equation of your NN.
In order to do that, I have modified your code and added fixed values for your Y features and Y labels. I'm doing that in order to show the whole calculation step by step so that next time you can do it yourself.
Based on all the information you have provided, it seems that you have
Let's start with the normalization layer. For normalization layers, it is kind "strange" in my opinion to use the term "weight". The weights are basically the mean and variance which will be applied to each input in order to normalize the data.
I wil call the 2 input features x0 and x1
if you run my code (which is your code with my fixed data), you will see that the weights for the normalization layer are
[5. 4.6] [ 5.4 11.24]
It means that the means for your [x0 x1] columns are [5. 4.6] and the variances are [5.4 11.24]
Can we verify that? Yes, we can. Let's check for x0.
[1,4,8,7,3,6,6,5,2,8,5]
mean = 5
stddev = 2.323790008
variance = 5.4 ( variance = stddev^2)
As you can see, it matches the "weights" of the normalization layer.
As data is pushed thru the normalization layer, each value will be normalized based on x' = (x-mean)/stddev ( stddev, not variance )
You can check that by applying the normalization to the data. In the code, if you run this 2 lines
normalized_data = normalizer(fundingFeatures)
print(normalized_data)
You will get
[[-1.7213259 1.31241 ]
[-0.43033147 1.014135 ]
[ 1.2909944 0.41758505]
[ 0.86066294 -0.47723997]
[-0.86066294 -1.07379 ]
[ 0.43033147 1.31241 ]
[ 0.43033147 -1.07379 ]
[ 0. -1.07379 ]
[-1.2909944 0.71586 ]
[ 1.2909944 -1.07379 ]]
Let's verify the first number.
x0[0] = 1
x0'[0] = (1-5)/2.323790008 = -1.7213 ( it does match)
At this point, we should be able to write the equations for the normalization layer
y[0]' = (x0-5)/2.323790008 # (x-mean)/stddev
y[1]' = (x1-4.6)/3.352610923
Now, these 2 outputs will be inject in the next layer. Remember, you have a Dense layer and therefore it is fully connected. It means that both values will be inject in the single neuron.
These lines show the value of both weights and bias for the Dense layer.
weights = model.layers[1].get_weights()[0]
biases = model.layers[1].get_weights()[1]
print(weights)
print(biases)
[[-0.12915221]
[-0.41322172]]
[0.32663438]
A neuron multiplies each input by a given weight, adds all results with the bias. Let's modify y[0]' and y[1]' to include the weights.
y[0]' = (x0-5)/2.323790008)* -0.12915221
y[1]' = (x1-4.6)/3.352610923 * -0.41322172
We are close, we just need to sum up these 2 and add the bias
y' = ((x0-5)/2.323790008)* -0.12915221 + (x1-4.6)/3.352610923 * -0.41322172 + 0.32663438
Since you don't have an activation function, we can stop here.
How can we verify if the formula is right? Let's use the model to predict the label for a random input and see if it matches the result we get when we put the same values in our equation.
First, let's run a model prediction for [4,5]
print(model.predict( [[4,5]] ))
[[0.3329112]]
Now, let's plug the same inputs to our equation
y' = (((4-5)/2.323790008)* -0.12915221) + ((5-4.6)/3.352610923 * -0.41322172) + 0.32663438
y' = 0.332911
It seems that we are good. I cut some precisions just be make my life easier.
Here is the function for your model. Just replace my numbers with your numbers.
y' = ((x0-5)/2.323790008)* -0.12915221 + (x1-4.6)/3.352610923 * -0.41322172 + 0.32663438
And here is the code. I have also added tensorboard so you can verify yourself what I have said here.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.layers.experimental import preprocessing
from matplotlib import pyplot as plt
import numpy as np
import datetime
fundingFeatures = tf.constant([[1, 9], [4, 8], [8, 6], [7 ,3], [3 ,1], [6, 9], [6, 1], [5, 1], [2, 7], [8, 1]], dtype=tf.int32)
fundingLabels = tf.constant([ 0.8160469, -0.05249139, 1.1515405, 1.0792135, 0.80369186, -1.7353221, 1.0092108, 0.19228514, -0.10366996, 0.10583907])
normalizer = preprocessing.Normalization()
normalizer.adapt(fundingFeatures)
normalized_data = normalizer(fundingFeatures)
print(normalized_data)
print("Features mean raw: %.2f" % (fundingFeatures[:,0].numpy().mean()))
print("Features std raw: %.2f" % (fundingFeatures[:,0].numpy().std()))
print("Features mean raw: %.2f" % (fundingFeatures[:,1].numpy().mean()))
print("Features std raw: %.2f" % (fundingFeatures[:,1].numpy().std()))
print("Features mean: %.2f" % (normalized_data.numpy().mean()))
print("Features std: %.2f" % (normalized_data.numpy().std()))
model = tf.keras.Sequential([
normalizer,
layers.Dense(units=1)
])
model.compile(loss = tf.losses.MeanSquaredError(),
optimizer = tf.keras.optimizers.SGD(
learning_rate=0.06, momentum=0.0, nesterov=True, name="SGD",
))
log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.summary()
print('--------------')
weights = model.layers[0].get_weights()[0]
biases = model.layers[0].get_weights()[1]
print('--------------')
model.fit(fundingFeatures, fundingLabels, epochs=1000, callbacks=[tensorboard_callback])
weights = model.layers[0].get_weights()[0]
biases = model.layers[0].get_weights()[1]
print(weights)
print(biases)
print ("\n")
weights = model.layers[1].get_weights()[0]
biases = model.layers[1].get_weights()[1]
print(weights)
print(biases)
print('\n--------- Prediction ------')
print(model.predict( [[4,5]] ))