How to add skip connection between convolutional layers in Keras

Question

I would like to add a skip connection between residual blocks in keras. This is my current implementation, which does not work because the tensors have different shapes.

The function looks like this:

def build_res_blocks(net, x_in, num_res_blocks, res_block, num_filters, res_block_expansion, kernel_size, scaling):
    net_next_in = net
    for i in range(num_res_blocks):
        net = res_block(net_next_in, num_filters, res_block_expansion, kernel_size, scaling)
    
        # net tensor shape: (None, None, 32)
        # x_in tensor shape: (None, None, 3)
        # Error here, net_next_in should be in the shape of (None, None, 32) to be fed into next layer
        net_next_in = Add()([net, x_in]) 

    return net

But I get

ValueError: Operands could not be broadcast together with shapes (None, None, 32) (None, None, 3)

How to add or merge these tensors into the correct shape (None, None, 32)? If this is not the correct approach, how could you achieve the intended result?

This is what the res_block looks like:

def res_block(x_in, num_filters, expansion, kernel_size, scaling):
    x = Conv2D(num_filters * expansion, kernel_size, padding='same')(x_in)
    x = Activation('relu')(x)
    x = Conv2D(num_filters, kernel_size, padding='same')(x)
    x = Add()([x_in, x])
return x

Answer 1

You cant add tensors of different shape. You could concatenate them with keras.layers.Concatenate, but this would leave you with a tensor of shape [None, None, 35].

Alternatively, have a look at the Resnet50 implementation in Keras. Their residual block features a 1x1xC convolution in the shortcut for those cases where the dimensions to be added are different.