Calling merge layer on batch data in Keras

Question

Is it possible to average the output of a previous layer over the training batch"

I have tried calling the Keras Average layer on the output of a Dense layer.

Here's a simple example.

from keras.models import  Model
from keras import layers
from keras import Input
from keras.utils import plot_model

input_tensor = layers.Input(shape=(784,))
output = layers.Dense(10,)(input_tensor)
average = layers.Average()(output)
avgout = Model(input_tensor, avgout)
avgout.summary()

The "avgout" layer should give the mean output of the output layer. The result:

ValueError                                Traceback (most recent call last)
<ipython-input-7-9d5576113651> in <module>
      6 input_tensor = layers.Input(shape=(784,))
      7 output = layers.Dense(10,)(input_tensor)
----> 8 average = layers.Average()(output)
      9 avgout = Model(input_tensor, avgout)
     10 avgout.summary()

~/anaconda3/lib/python3.7/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
    429                                          'You can build it manually via: '
    430                                          '`layer.build(batch_input_shape)`')
--> 431                 self.build(unpack_singleton(input_shapes))
    432                 self.built = True
    433 

~/anaconda3/lib/python3.7/site-packages/keras/layers/merge.py in build(self, input_shape)
     66         # Used purely for shape validation.
     67         if not isinstance(input_shape, list):
---> 68             raise ValueError('A merge layer should be called '
     69                              'on a list of inputs.')
     70         if len(input_shape) < 2:

ValueError: A merge layer should be called on a list of inputs.

Answer 1

The Average layer in keras computer the average of multiple tensors, but not the mean of one tensor.

What you can do is using keras backend mean :

from keras import backend as K
from keras.models import  Model
from keras import layers
from keras import Input
from keras.utils import plot_model

def mean(input):
    return K.mean(input, axis=1)

input_tensor = layers.Input(shape=(784,))
output = layers.Dense(10,)(input_tensor)
average = layers.Lambda(mean, input_shape=(10,))(output)
avgout = Model(input_tensor, average)
avgout.summary()