Can I use a 3D input on a Keras Dense Layer?

Question

As an exercise I need to use only dense layers to perform text classifications. I want to leverage words embeddings, the issue is that the dataset then is 3D (samples,words of sentence,embedding dimension). Can I input a 3D dataset into a dense layer?

Thanks

Answer 1

As stated in the keras documentation you can use 3D (or higher rank) data as input for a Dense layer but the input gets flattened first:

Note: if the input to the layer has a rank greater than 2, then it is flattened prior to the initial dot product with kernel.

This means that if your input has shape (batch_size, sequence_length, dim), then the dense layer will first flatten your data to shape (batch_size * sequence_length, dim) and then apply a dense layer as usual. The output will have shape (batch_size, sequence_length, hidden_units). This is actually the same as applying a Conv1D layer with kernel size 1, and it might be more explicit to use a Conv1D layer instead of a Dense layer.