Search code examples
python-3.xkeraskeras-layer

Keras custom layer/constraint to implement equal weights

I would like to create a layer in Keras such that:

y = Wx + c

where W is a block matrix with the form:

enter image description here

A and B are square matrices with elements:

enter image description here enter image description here

and c is a bias vector with repeated elements:

enter image description here

How can I implement these restrictions? I was thinking it could either be implemented in the MyLayer.build() when initializing weights or as a constraint where I can specify certain indices to be equal but I am unsure how to do so.

Solution

  • You can define such W using Concatenate layer.

    import keras.backend as K
from keras.layers import Concatenate

A = K.placeholder()
B = K.placeholder()

row1 = Concatenate()([A, B])
row2 = Concatenate()([B, A])
W = Concatenate(axis=1)([row1, row2])

    Example evaluation:

    import numpy as np

get_W = K.function(outputs=[W], inputs=[A, B])
get_W([np.eye(2), np.ones((2,2))])

    Returns

    [array([[1., 0., 1., 1.],
        [0., 1., 1., 1.],
        [1., 1., 1., 0.],
        [1., 1., 0., 1.]], dtype=float32)]

    To figure out exact solution you can use placeholder's shape argument. Addition and multiplication are quite straightforward.