How to pass a sparse tensor to the Dense Layer in TF 2.0?

Question

I am using TF 2.0.

WORKING:

from tensorflow.keras import layers

inputs = layers.Input(shape=(256,), sparse=False, name='name_sparse')
x = layers.Dense(32, name="my_layer")(inputs)
print(x)

Output: Tensor("my_layer/Identity:0", shape=(None, 32), dtype=float32)

NOT WORKING:

If I change sparse to True in the above code, the output changes to:

ValueError: The last dimension of the inputs to Dense should be defined. Found None.

How can I pass a sparse tensor to the Dense layer in TF2.0. It works well in TF1.14.

Answer 1

This happens because when input tensor is sparse shape of this tensor evaluates to (None,None) instead of (256,)

inputs = layers.Input(shape=(256,), sparse=True, name='name_sparse')
print(inputs.shape) 
# output: (?, ?)

This also seems to be an open issue.
One solution is to write custom layer sub-classing Layer class (Refer this).

As a work-around (tested on tf-gpu 2.0.0) adding batch-size in input layer works fine:

from tensorflow.keras import layers
inputs = layers.Input(shape=(256,), sparse=True, name='name_sparse', batch_size=32)
print(inputs.shape) # (32, 256)
x = layers.Dense(32, name="my_layer")(inputs)
print(x) # Tensor("my_layer_10/BiasAdd:0", shape=(32, 32), dtype=float32)