Keras: Feeding in part of previous layer to next layer, in CNN

Question

I am trying to feed in the individual kernel outputs of the previous layer to a new conv filter, to get the next layer. To do that, I tried passing each of the kernel outputs through a Conv2D, by calling them by their index. The function I used is:

def modification(weights_path=None, classes=2):

    ###########
    ## Input ##
    ###########

    ### 224x224x3 sized RGB Input
    inputs = Input(shape=(224,224,3))

    #################################
    ## Conv2D Layer with 5 kernels ##
    #################################

    k = 5
    x = Conv2D(k, (3,3), data_format='channels_last', padding='same', name='block1_conv1')(inputs)
    y = np.empty(k, dtype=object)
    for i in range(0,k):
        y[i] = Conv2D(1, (3,3), data_format='channels_last', padding='same')(np.asarray([x[i]]))
    y = keras.layers.concatenate([y[i] for i in range (0,k)], axis=3, name='block1_conv1_loc')
    out = Activation('relu')(y)
    print ('Output shape is, ' +str(out.get_shape()))

    ### Maxpooling(2,2) with a stride of (2,2)
    out = MaxPooling2D((2,2), strides=(2,2), data_format='channels_last')(out)

    ############################################
    ## Top layer, with fully connected layers ##
    ############################################

    out = Flatten(name='flatten')(out)
    out = Dense(4096, activation='relu', name='fc1')(out)
    out = Dropout(0.5)(out)
    out = Dense(4096, activation='relu', name='fc2')(out)
    out = Dropout(0.5)(out)
    out = Dense(classes, activation='softmax', name='predictions')(out)

    if weights_path:
        model.load_weights(weights_path)

    model = Model(inputs, out, name='modification')

    return model

But this is not working, and is throwing the following error:

Traceback (most recent call last):
  File "sim-conn-edit.py", line 137, in <module>
    model = modification()
  File "sim-conn-edit.py", line 38, in modification
    y[i] = Conv2D(1, (3,3), data_format='channels_last', padding='same')(np.asarray([x[i]]))
  File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 511, in __call__
    self.assert_input_compatibility(inputs)
  File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 408, in assert_input_compatibil
ity
    if K.ndim(x) != spec.ndim:
  File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 437, in ndim
    dims = x.get_shape()._dims
AttributeError: 'numpy.ndarray' object has no attribute 'get_shape'

I fed in layer x[i] as [ x[i] ] to meet the dimention requirements of a Conv2D layer. Any help with solving this problem will be deeply appreciated!

Answer 1

After posting this and this questions in StackOverflow, and some personal exploring, I came up with a solution. One can possibly do this with Lambda layers; by calling a Lambda layer to extract a sub part of the previous layer. For example, if the Lambda function is defined as,

def layer_slice(x,i):
    return x[:,:,:,i:i+1]

and then, called as,

k = 5
x = Conv2D(k, (3,3), data_format='channels_last', padding='same', name='block1_conv1')(inputs)
y = np.empty(k, dtype=object)
for i in range(0,k):
    y[i] = Lambda(layer_slice, arguments={'i':i})(x)
    y[i] = Conv2D(1,(3,3), data_format='channels_last', padding='same')(y[i])
y = keras.layers.concatenate([y[i] for i in range (0,k)], axis=3, name='block1_conv1_loc')
out = Activation('relu')(y)
print ('Output shape is, ' +str(out.get_shape()))

it should effectively feed in individual kernel outputs to a new Conv2D layer. The layer shapes and corresponding number of trainable parameters being obtained from model.summary() matches the expectation. Thanks to Daniel for pointing out that Lambda layers cannot have trainable weights.