Search code examples
tensorflowkerasconv-neural-network

Why does TensorFlow Conv2D have two weights matrices?

I have a tf.keras.layers.Conv2D constructed like so:

>>> conv2d_layer = tf.keras.layers.Conv2D(filters=128, kernel_size=(3, 3), strides=2)

For reference that layer is part of a network where the prior layer is prior_layer = Conv2D(filters=64, kernel_size=(3, 3), strides=2).

When I call conv2d_layer.get_weights(), it returns a list with two entries:

>>> [w.shape for w in conv2d_layer.get_weights()]
[(3, 3, 64, 128), (128,)]

Why are there two np.ndarrays in conv2d_layer.get_weights()? What are their respective meanings?

Solution

  • The first shape is for the weights of your conv2D, and the second one is the bias for the same layer, which is represented by a vector.

    Looking at the documentation, you can see

    For example, a Dense layer returns a list of two values: the kernel matrix and the bias vector. These can be used to set the weights of another Dense layer: