Search code examples
pythonneural-networkkeraskeras-layer

Custom connections between layers Keras

I would like to manually define connections in neural network between layers using keras with Python. By default connections are beween all pairs of neurons. I need to make connections as in picture below.

required architecture

How can I be done in Keras?

Solution

  • You can use the functional API model and separate four distinct groups:

    from keras.models import Model
from keras.layers import Dense, Input, Concatenate, Lambda

inputTensor = Input((8,))

    First, we can use lambda layers to split this input in four:

    group1 = Lambda(lambda x: x[:,:2], output_shape=((2,)))(inputTensor)
group2 = Lambda(lambda x: x[:,2:4], output_shape=((2,)))(inputTensor)
group3 = Lambda(lambda x: x[:,4:6], output_shape=((2,)))(inputTensor)
group4 = Lambda(lambda x: x[:,6:], output_shape=((2,)))(inputTensor)

    Now we follow the network:

    #second layer in your image
group1 = Dense(1)(group1)
group2 = Dense(1)(group2)
group3 = Dense(1)(group3)   
group4 = Dense(1)(group4)

    Before we connect the last layer, we concatenate the four tensors above:

    outputTensor = Concatenate()([group1,group2,group3,group4])

    Finally the last layer:

    outputTensor = Dense(2)(outputTensor)

#create the model:
model = Model(inputTensor,outputTensor)

    Beware of the biases. If you want any of those layers to have no bias, use use_bias=False.

    Old answer: backwards

    Sorry, I saw your image backwards the first time I answered. I'm keeping this here just because it's done...

    from keras.models import Model
from keras.layers import Dense, Input, Concatenate

inputTensor = Input((2,))

#four groups of layers, all of them taking the same input tensor
group1 = Dense(1)(inputTensor)
group2 = Dense(1)(inputTensor)
group3 = Dense(1)(inputTensor)   
group4 = Dense(1)(inputTensor)

#the next layer in each group takes the output of the previous layers
group1 = Dense(2)(group1)
group2 = Dense(2)(group2)
group3 = Dense(2)(group3)
group4 = Dense(2)(group4)

#now we join the results in a single tensor again:
outputTensor = Concatenate()([group1,group2,group3,group4])

#create the model:
model = Model(inputTensor,outputTensor)