How to create a model with multiple shared layers in Keras Functional API?

Question

I would like to have a model with 2 inputs, several hidden layers with shared weights, then separate output layers.

I have seen this question and its accepted answer: Share weights between two dense layers in keras . This is exactly what I would want to achieve, just with multiple shared dense layers.

Basically, this is what they do:
(I modified it a little to have 2 separate output layers)

ip_shape1 = tf.keras.layers.Input(shape=(5,))
ip_shape2 = tf.keras.layers.Input(shape=(5,))

dense = tf.keras.layers.Dense(1, activation="sigmoid", kernel_initializer="ones")

op1 = dense(ip_shape1)
op2 = dense(ip_shape2)

op1 = tf.keras.layers.Dense(1,activation=tf.nn.sigmoid)(op1)
op2 = tf.keras.layers.Dense(1,activation=tf.nn.sigmoid)(op2)

model = tf.keras.models.Model(inputs=[ip_shape1, ip_shape2], outputs=[op1,op2])

I would like to do the same, just with 2 shared hidden layers:

ip_shape1 = tf.keras.layers.Input(shape=(5,))
ip_shape2 = tf.keras.layers.Input(shape=(5,))

dense = tf.keras.layers.Dense(1, activation="sigmoid", kernel_initializer="ones", input_shape=(5,))
dense = tf.keras.layers.Dense(1, activation="sigmoid", kernel_initializer="ones")(dense)

op1 = dense(ip_shape1)
op2 = dense(ip_shape2)

op1 = tf.keras.layers.Dense(1,activation=tf.nn.sigmoid)(op1)
op2 = tf.keras.layers.Dense(1,activation=tf.nn.sigmoid)(op2)

model = tf.keras.models.Model(inputs=[ip_shape1, ip_shape2], outputs=[op1,op2])

But when I try to do this, I get an error:

TypeError: Inputs to a layer should be tensors. Got: <tensorflow.python.keras.layers.core.Dense object at 0x7f7286dc7c70>

Answer 1

The error occurs in the following line:

dense = tf.keras.layers.Dense(1, activation="sigmoid", kernel_initializer="ones")(dense)

Essentially, you are calling the dense layer with dense (another Keras layer). Instead, the layer tf.keras.layers.Dense expects a tensor as input.

---

I presume that you want to compose two shared dense layers. This can be achieved as follows:

dense_1 = tf.keras.layers.Dense(1, activation="sigmoid", kernel_initializer="ones", input_shape=(5,))
dense_2 = tf.keras.layers.Dense(1, activation="sigmoid", kernel_initializer="ones")

op1 = dense_1(ip_shape1)
op1 = dense_2(op1)

op2 = dense_1(ip_shape2)
op2 = dense_2(op2)

NOTE: Not tested.