keras Bidirectional layer using 4 dimension data

Question

I'am designing keras model for classification based on article data.

I have data with 4 dimension as follows

[batch, article_num, word_num, word embedding size]

and i want to feed each (word_num, word embedding) data into keras Bidirectional layer

in order to get result with 3 dimension as follows.

[batch, article_num, bidirectional layer output size]

when i tried to feed 4 dimension data for testing like this

inp = Input(shape=(article_num, word_num, ))
# dims = [batch, article_num, word_num]

x = Reshape((article_num * word_num, ), input_shape = (article_num, word_num))(inp)
# dims = [batch, article_num * word_num]

x = Embedding(word_num, word_embedding_size, input_length = article_num * word_num)(x)
# dims = [batch, article_num * word_num, word_embedding_size]

x = Reshape((article_num , word_num, word_embedding_size), 
             input_shape = (article_num * word_num, word_embedding_size))(x)
# dims = [batch, article_num, word_num, word_embedding_size]

x = Bidirectional(CuDNNLSTM(50, return_sequences = True), 
                  input_shape=(article_num , word_num, word_embedding_size))(x)

and i got the error

ValueError: Input 0 is incompatible with layer bidirectional_12: expected ndim=3, found ndim=4

how can i achieve this?

Answer 1

If you don't want it to touch the article_num dimension, you can try using a TimeDistributed wrapper. But I'm not certain that it will be compatible with bidirectional and other stuff.

inp = Input(shape=(article_num, word_num))    

x = TimeDistributed(Embedding(word_num, word_embedding_size)(x))

#option 1
#x1 shape : (batch, article_num, word_num, 50)
x1 = TimeDistributed(Bidirectional(CuDNNLSTM(50, return_sequences = True)))(x)

#option 2
#x2 shape : (batch, article_num, 50)
x2 = TimeDistributed(Bidirectional(CuDNNLSTM(50)))(x)

Hints:

• Don't use input_shape everywhere, you only need it at the Input tensor.
• You probably don't need any of the reshapes if you also use a TimeDistributed in the embedding.
• If you don't want word_num in the final dimension, use return_sequences=False.