Search code examples
neural-networkkerasbackpropagationkeras-layerloss-function

Behaviour of Model.fit with shared layers in keras


I have the following model:

enter image description here

sharedLSTM1 = LSTM((data.shape[1]), return_sequences=True)
sharedLSTM2 = LSTM(data.shape[1])
def createModel(dropoutRate=0.0, numNeurons=40, optimizer='adam'):
    inputLayer = Input(shape=(timesteps, data.shape[1]))
    sharedLSTM1Instance = sharedLSTM1(inputLayer)
    sharedLSTM2Instance =  sharedLSTM2(sharedLSTM1Instance)
    dropoutLayer = Dropout(dropoutRate)(sharedLSTM2Instance)
    denseLayer1 = Dense(numNeurons)(dropoutLayer)
    denseLayer2 = Dense(numNeurons)(denseLayer1)
    outputLayer = Dense(1, activation='sigmoid')(denseLayer2)
    return (inputLayer, outputLayer)

inputLayer1, outputLayer1 = createModel()
inputLayer2, outputLayer2 = createModel()
model = Model(inputs=[inputLayer1, inputLayer2], outputs=[outputLayer1, outputLayer2])
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

What will be the behaviour of model.fit([data1, data2], [labels1, labels2]) in this model. Will it alternatively train the two NNs for each epoch? Or will it completely train one network, and then train the other? Or maybe some other way?


Solution

  • It will train the only existing network at once.

    You don't have two models, you have one model only. This model will be trained.

    Data1 and Data2 will be fed simultaneously.
    The loss function will be applied to both outputs, and both will backpropagate.