I am trying to create a custom layer in keras, but I'm coming across a strange issue. When I'm summing the tensors before returning the answer, the dimensions change. This happens when I sum the bias weights to the two tensors, see the code below.
summation = tf.math.add(biTensorVector, normalTensorVector)
print('summation1 ',summation)
summation = tf.math.add(summation,self.b)
print('summation2 ',summation)
which gives
summation1 Tensor("sequential/ntn/Add:0", shape=(5, None), dtype=float32)
summation2 Tensor("sequential/ntn/Add_1:0", shape=(5, 5), dtype=float32)
which surely should give an output shape (5,None)?
self.b is initialised with:
self.b = self.add_weight(name='b',shape=(self.k,),
initializer='zeros',
trainable=True)
Shouldn't the weight self.b broadcast to be the shape (5,None) then added to the summation, rather than removing the None dimension? Any clarification would be much appreciated.
The None size is from the initialisation of the layer, supposedly if it works with None it ought to work with any number of samples I assume? Hence why I'm quite confused.
I should add I have just tried the + operator and I have the same issue.
Shapes of biTensorVector, normalTensorVector are probably (5, None). So adding them gives (5, None) because the second dimension is unknown.
Shape of self.b is (None, 5). Adding (5, None) and (None, 5) always get (5, 5) (or error). So keras knows that the result of the second addition is (5, 5).