Search code examples
python-3.xtensorflowkeraskeras-layerresnet

How does the syntax X = Add() ([tensor1, tensor2]) exactly work in using keras.layers.add to construct a ResNET?


In some code for constructing a ResNET that I was viewing, I found the following syntax:

.
.
.
# Save the input value. You'll need this later to add back to the main path. 
X_shortcut = X
.
.
.
# Final step: Add shortcut value to main path, and pass it through a RELU activation (≈2 lines)
X = Add()([X, X_shortcut])
X = Activation('relu')(X)

Ref: https://github.com/priya-dwivedi/Deep-Learning/blob/master/resnet_keras/Residual_Networks_yourself.ipynb

I understand that [X, X_shortcut] is the input list to the Add() layer from keras.layers where X and X_shortcut are symbolic tensors. However, I don't the understand the syntax. Could someone please explain to me how the input is being passed here, and why the tensor list is not simply an argument to the Add() function?


Solution

  • The Add() is the constructor for the a keras layers class. We first create one of these (callable) objects then call it with the list of inputs.

    It's shorthand for something like this:

    addition_layer = Add()
    activation_layer = Activation('relu')
    
    .
    .
    .
    added = addition_layer([X, X_shortcut])
    after_activation = activation_layer(added)