Search code examples
pythonmachine-learningkeraskeras-layer

Keras InputLayer exists in models created by Functional API but not in Sequential API?


I'm trying to understand the relationship between the Sequential API (i.e. Sequential()) and the Functional API (i.e. Model() for setting up neural networks in Keras. In particular, I'm confused by the existence of an InputLayer object in a model generated using Functional API and the absence of any counterpart in the Sequential version. Are the two versions shown below equivalent? Is the InputLayer object just a do-nothing placeholder? If not, what would have to be done to make the models equivalent?

from keras.layers import Input, Dense
from keras.models import Model,Sequential

N_in = 10
N_hidden = 10
N_out = 10

# Using Model API
input = Input(shape=(N_in,))
hidden = Dense(N_hidden)(input)
output = Dense(N_out)(hidden)
model1 = Model(input, output)

# Using Sequential API
model2= Sequential()
model2.add(Dense(N_hidden, input_dim=N_in))
model2.add(Dense(N_out))

for i in range(len(model1.layers)):
    print(model1.layers[i])

keras.engine.input_layer.InputLayer object at 0xb333b8c88

keras.layers.core.Dense object at 0xb333b87b8>

keras.layers.core.Dense object at 0xb333b8b00>

for i in range(len(model2.layers)):
    print(model2.layers[i])

keras.layers.core.Dense object at 0xb331eddd8

keras.layers.core.Dense object at 0xb3333dcc0


Solution

  • Are the two versions shown below equivalent?

    Yes, these two ways define the same model. Once your model architecture is defined, all following steps are the same whether you used a Sequential() or the Model().

    Is the InputLayer object just a do-nothing placeholder?

    Yes, InputLayer object is a do-nothing placeholder. If you use summary(), you will find that its param number is 0.

    print(model1.summary())
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    input_1 (InputLayer)         (None, 10)                0         
    _________________________________________________________________
    dense_1 (Dense)              (None, 10)                110       
    _________________________________________________________________
    dense_2 (Dense)              (None, 10)                110       
    =================================================================
    Total params: 220
    Trainable params: 220
    Non-trainable params: 0
    

    Sequential() is only used for linear stacks of layers. Model() is used for directed acyclic graphs of layers, allowing to build completely arbitrary architectures(like several inputs and several outputs).

    The output model.layers you see is different because layers have different instances. They are different even on different machines, but they work the same way.