Search code examples
pythontensorflowkerasdeep-learning

Implementing simple sequential model by keras

I am a newbie to Tensorflow and I am trying to follow the steps detailed on https://www.tensorflow.org/guide/keras/sequential_model

#The Sequential model_1
#https://www.tensorflow.org/guide/keras/sequential_model

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

if (__name__=="__main__"):

    # Define Sequential model_1 with 3 layers
    model_1 = keras.Sequential(
        [
            layers.Dense(2, activation="relu", name="layer_1"),
            layers.Dense(3, activation="relu", name="layer_2"),
            layers.Dense(4, name="layer_3"),
        ]
    )
    # Call model_1 on a test input
    x = tf.ones((3, 3))
    y1 = model_1(x)

    model_1.layers
    ####################################################################

    # Create 3 discrete layers as above
    x = tf.ones((3, 3))
    layer1 = layers.Dense(2, activation="relu", name="layer1")
    layer2 = layers.Dense(3, activation="relu", name="layer2")
    layer3 = layers.Dense(4, name="layer3")
    y2 = layer3(layer2(layer1(x)))
    ####################################################################
    print('y1=', y1)
    print('y2=', y2)

According to the TF guide, y1 and y2 should be identical. But here's the output of the program. Does anyone have any idea what's wrong? Why are y1 and y2 different?

y1= tf.Tensor([[0. 0. 0. 0.][0. 0. 0. 0.][0. 0. 0. 0.]], shape=(3, 4), dtype=float32)

y2= tf.Tensor([[ 0.16045676  0.02784279  0.2826522  -0.2760685 ][ 0.16045676  0.02784279  0.2826522  -0.2760685 ][ 0.16045676  0.02784279  0.2826522  -0.2760685 ]], shape=(3, 4), dtype=float32)

If the code is modified as

model_1 = keras.Sequential(
        [
            layers.Dense(2, activation="relu", name="layer1"),
            layers.Dense(3, activation="relu", name="layer2"),
            layers.Dense(4, name="layer3"),
        ]
    )

The output will be like this

y1= tf.Tensor([[-0.56710994  0.4444507   0.09539947  0.7368923 ][-0.56710994  0.4444507   0.09539947  0.7368923 ][-0.56710994  0.4444507   0.09539947  0.7368923 ]], shape=(3, 4), dtype=float32)

y2= tf.Tensor([[-0.05642081  0.04768222 -0.01716159  0.00080794][-0.05642081  0.04768222 -0.01716159  0.00080794][-0.05642081  0.04768222 -0.01716159  0.00080794]], shape=(3, 4), dtype=float32)

Does it matter which name is assigned to the layer??!!

Solution

  • Actually those are different layers with different weights, so the output is different. If you want to get the same output, you should use those layers to build the model.

    Fixed code:

    import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

layer1 = layers.Dense(2, activation="relu", name="layer1")
layer2 = layers.Dense(3, activation="relu", name="layer2")
layer3 = layers.Dense(4, name="layer3")

model_1 = keras.Sequential([layer1, layer2, layer3])

x = tf.ones((3, 3))

y1 = model_1(x)
y2 = layer3(layer2(layer1(x)))

print('y1=', y1)
print('y2=', y2)

    Output:

    y1= tf.Tensor(
[[-1.0433494   1.3083773   0.47221014 -0.11679913]
 [-1.0433494   1.3083773   0.47221014 -0.11679913]
 [-1.0433494   1.3083773   0.47221014 -0.11679913]], shape=(3, 4), dtype=float32)
y2= tf.Tensor(
[[-1.0433494   1.3083773   0.47221014 -0.11679913]
 [-1.0433494   1.3083773   0.47221014 -0.11679913]
 [-1.0433494   1.3083773   0.47221014 -0.11679913]], shape=(3, 4), dtype=float32)