Appending tensors in Keras

Question

I have a for loop in my keras model as follow:

 for t in range(Ty):
   ....
   ....
   ....

   out = Dense(num_dec_tokens, activation='softmax')(x)  # out.shape = (?, num_dec_tokens) 

Is there a way that I can append the tensor 'out' for Ty times (for example using a lambda layer) without using a list, i.e., not using: outputs = [], then ... outputs.append(out)?

If so, is there a way that I can change the appended tensors into a shape of (?, Ty, num_dec_tokens) instead of (Ty, ?, num_dec_tokens)?

Thanks...

Answer 1

One approach is to reshape the outputs.

outputs = []

for i in range(Ty):
    out = Dense(3, activation="softmax")(x)
    outputs.append(out)

output = Concatenate()(outputs)
output = Reshape([Ty,3])(output)

Using RepeatVector we can convert (None, num_dec_tokens) => (None, Ty, num_dec_tokens)

>>> dense = Dense(num_dec_tokens, activation="softmax")(x)
>>> out = RepeatVector(Ty)(dense)

In your case you will be learning Ty Dense layers, but not when using RepeatVector.