Tensorflow/Keras Conv2D layers with padding='SAME' behave strangely

Question

My question:

A straightforward experiment that I conducted showed that using padding='SAME' in a conv2d layer in Keras/TF is different from using padding='VALID' with a preceding zero-padding layer.

1. How is that possible?
2. Does Keras/TF pads zeros symmetrically around the tensor?

Explanation of the experiment - just if you're interested in reading further:

I used the onnx2keras package to convert my Pytorch model into keras/TF.

When onnx2keras encounters a convolutional layer with padding > 0 in the ONNX model, it translates it to Keras' Conv2D with valid padding (i.e., no padding!), preceded by Keras' ZeroPadding2D layer. This works very well and returns outputs that are identical to those produced by the Pytorch network.

I yet thought it was strange that it didn't simply used padding='SAME', as most of the references say that Keras/TF use zero padding, just like Pytorch does.

Nevertheless, I patched onnx2keras and made it produce me Conv2D layers with padding='SAME' rather than the existing solution of 'VALID' padding with a preceding zero-padding layer. This made the resulting model return different outputs than the one with the zero-padding layer, and of course different from my Pytorch model, which was identical until the patch.

Answer 1

padding='Same' in Keras means padding is added as required to make up for overlaps when the input size and kernel size do not perfectly fit.

Example of padding='Same':

# Importing dependency
import keras
from keras.models import Sequential
from keras.layers import Conv2D

# Create a sequential model
model = Sequential()

# Convolutional Layer
model.add(Conv2D(filters=24, input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2) ,padding='Same'))

# Model Summary
model.summary()

Output of the code -

Model: "sequential_20"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_28 (Conv2D)           (None, 3, 3, 24)          120       
=================================================================
Total params: 120
Trainable params: 120
Non-trainable params: 0
_________________________________________________________________

Pictorial Representation: Below image shows how the padding for the input (input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2)) when padding='Same'.

------------------------------------------------------------------------------------------------------------------

padding='Valid' in Keras means no padding is added.

Example of padding='Valid': Have used same input for Conv2D that we used above for padding = 'Same' .i.e. (input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2))

# Importing dependency
import keras
from keras.models import Sequential
from keras.layers import Conv2D

# Create a sequential model
model = Sequential()

# Convolutional Layer
model.add(Conv2D(filters=24, input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2) ,padding='Valid'))

# Model Summary
model.summary()

Output of the code -

Model: "sequential_21"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_29 (Conv2D)           (None, 2, 2, 24)          120       
=================================================================
Total params: 120
Trainable params: 120
Non-trainable params: 0
_________________________________________________________________

Pictorial Representation: Below image shows there is no padding added for the input (input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2)) when padding='Valid'.

------------------------------------------------------------------------------------------------------------------

Now lets try same code that we used for padding='Valid' for the input (input_shape=(6,6,1), kernel_size=(2,2), strides =(2,2)). Here padding='Valid' should behave same as padding='Same'.

Code -

# Importing dependency
import keras
from keras.models import Sequential
from keras.layers import Conv2D

# Create a sequential model
model = Sequential()

# Convolutional Layer
model.add(Conv2D(filters=24, input_shape=(6,6,1), kernel_size=(2,2), strides =(2,2) ,padding='Valid'))

# Model Summary
model.summary()

Output of the code -

Model: "sequential_22"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_30 (Conv2D)           (None, 3, 3, 24)          120       
=================================================================
Total params: 120
Trainable params: 120
Non-trainable params: 0
_________________________________________________________________