The performance of a model with different output loss configurations in Sequential and Functional APIs differ in Keras

Question

I am getting two very different results from the model coded in Sequential and Function APIs. What could go wrong with my code?

I started my code with the following:

np.random.seed(42)
tf.random.set_seed(42)
random.seed(42)

Then, I coded the following model in sequential API for 18 input features and 7 output:

model = keras.models.Sequential([
    Input(shape=18),
    Dense(1000, activation = "relu"),
    BatchNormalization(),
    Dense(1000, activation = "relu"),
    BatchNormalization(),
    Dense(7)
])
model.compile(optimizer=keras.optimizers.Adam(learning_rate=1e-4), loss=keras.losses.mse)
history = model.fit(X_tr, Y_tr, epochs=1000, validation_data=(X_val, Y_val))

I got the overall minimum validation MSE of 0.417.

However, then I had to go for Functional API to find the MSE separately for each output. So I tried to code the same above model as follows:

targets = ('output1', 'output2', 'output3', 'output4', 'output5', 'output6', 'output7')

inp = Input(shape=18)
hidden1 = Dense(1000, activation = "relu")(inp)
hidden2 = BatchNormalization()(hidden1)
hidden3 = Dense(1000, activation = "relu")(hidden2)
hidden4 = BatchNormalization()(hidden3)
out = [Dense(1, name = nn)(hidden4) for nn in targets ]
modelF = keras.Model(inputs = [inp], outputs = [out])

modelF.compile(optimizer=keras.optimizers.Adam(learning_rate=1e-4), loss=keras.losses.mse)
history = modelF.fit(X_tr, Y_tr, epochs=1000, validation_data=(X_val, Y_val))

But with this Functional API, I am getting the minimum MSE more than 1284 for each output. What is wrong with the code? How can I modify the Functional API code to get the same model if these aren't the same but still get 7 MSEs instead of just 1?

Answer 1

1. There is a logical error behind getting so high MSE in the Functional API code. Just replace the following line

   modelF = keras.Model(inputs = [inp], outputs = [out])

with the following:

modelF = keras.Model(inputs = [inp], outputs = out)

2. The loss will be provided for all the outputs separately in compilation stage i.e.,

   model.compile(optimizer=keras.optimizers.Adam(learning_rate=1e-4), loss=["mse", "mse", "mse", "mse", "mse", "mse", "mse"])

3. The output labels will also be provided separately for all stages of train, validation, and test stages. For the train and validation stages, you will write the following:

   historyF = modelF.fit(x = X_tr, y = [Y_tr[:,0], Y_tr[:,1], Y_tr[:,2], Y_tr[:,3], Y_tr[:,4], Y_tr[:,5], Y_tr[:,6] ], epochs=1000, validation_data=(X_val, [Y_val[:,0], Y_val[:,1], Y_val[:,2], Y_val[:,3], Y_val[:,4], Y_val[:,5], Y_val[:,6] ] ) )

This will give the normal MSEs similar to that of the Sequential API code.