I'm stack and I need the wisdom of stackoverflow.
I have a two inputs neural network implemented in Keras using the Functional API, the input shapes are:
X.shape, X_size.shape, y.shape
((123, 9), (123, 2), (123, 9, 10))
So, my problem is I want to get output shape from LSTMs have 3-D shape, in order to use my y tensor. I know, I can reshape my y to 2-D shape, but I want to use it as a 3-D array.
from keras.models import Model
from keras import layers
from keras import Input
# first input
list_input = Input(shape=(None,), dtype='int32', name='li')
embedded_list = layers.Embedding(100,90)(list_input)
encoded_list = layers.LSTM(4, name = "lstm1")(embedded_list)
# second input
size_input = Input(shape=(None,), dtype='int32', name='si')
embedded_size = layers.Embedding(100,10)(size_input)
encoded_size = layers.LSTM(4, name = "lstm2")(embedded_size)
# concatenate
concatenated = layers.concatenate([encoded_size, encoded_list], axis=-1)
answer = layers.Dense(90, activation='sigmoid', name = 'outpuy_layer')(concatenated)
model = Model([list_input, size_input], answer)
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=[f1])
Model summary:
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
si (InputLayer) (None, None) 0
____________________________________________________________________________________________________
li (InputLayer) (None, None) 0
____________________________________________________________________________________________________
embedding_16 (Embedding) (None, None, 10) 1000 si[0][0]
____________________________________________________________________________________________________
embedding_15 (Embedding) (None, None, 90) 9000 li[0][0]
____________________________________________________________________________________________________
lstm2 (LSTM) (None, 4) 240 embedding_16[0][0]
____________________________________________________________________________________________________
lstm1 (LSTM) (None, 4) 1520 embedding_15[0][0]
____________________________________________________________________________________________________
concatenate_8 (Concatenate) (None, 8) 0 lstm2[0][0]
lstm1[0][0]
____________________________________________________________________________________________________
outpuy_layer (Dense) (None, 90) 810 concatenate_8[0][0]
====================================================================================================
Total params: 12,570
Trainable params: 12,570
Non-trainable params: 0
One more time, the question is:
How to get output shape from LSTMs like (None, None, None/10)?
Keras ignores every timestep output except the last one by default, which creates a 2D array. To get a 3D array (meaning you get the output of every timestep), instantiate the layer with return_sequences set to True. In your case for example:
encoded_list = layers.LSTM(4, name = "lstm1", return_sequences=True)(embedded_list)