How to feed the output of PostgreSQL database from TensorFlow IO to tf.keras Sequential Model

Question

I was trying to learn how to read a SQL dataset into Tensorflow using this tutorial. Until the last step everything works fine!

dataset = tfio.experimental.IODataset.from_sql(
    query="SELECT pt08s1, nmhc, c6h6, co FROM AirQualityUCI;",
    endpoint=endpoint)

print(dataset.element_spec)
#{'pt08s1': TensorSpec(shape=(), dtype=tf.int32, name=None), 'nmhc': TensorSpec(shape=(), dtype=tf.float32, name=None), 'c6h6': TensorSpec(shape=(), dtype=tf.float32, name=None), 'co': TensorSpec(shape=(), dtype=tf.float32, name=None)}

I take only the first 10 records.

dataset = dataset.take(10)
for i in dataset.as_numpy_iterator():
    print(i)

# {'pt08s1': 1360, 'nmhc': 150.0, 'c6h6': 11.9, 'co': 2.6}
# {'pt08s1': 1292, 'nmhc': 112.0, 'c6h6': 9.4, 'co': 2.0}
# {'pt08s1': 1402, 'nmhc': 88.0, 'c6h6': 9.0, 'co': 2.2}
# {'pt08s1': 1376, 'nmhc': 80.0, 'c6h6': 9.2, 'co': 2.2}
# {'pt08s1': 1272, 'nmhc': 51.0, 'c6h6': 6.5, 'co': 1.6}
# {'pt08s1': 1197, 'nmhc': 38.0, 'c6h6': 4.7, 'co': 1.2}
# {'pt08s1': 1185, 'nmhc': 31.0, 'c6h6': 3.6, 'co': 1.2}
# {'pt08s1': 1136, 'nmhc': 31.0, 'c6h6': 3.3, 'co': 1.0}
# {'pt08s1': 1094, 'nmhc': 24.0, 'c6h6': 2.3, 'co': 0.9}
# {'pt08s1': 1010, 'nmhc': 19.0, 'c6h6': 1.7, 'co': 0.6}

Now, my y is co variable and the rest are independent variables. I want to create a very basic DNN regression model for demo purposes.

model = tf.keras.Sequential([
      tf.keras.layers.Dense(64, activation='relu'),
      tf.keras.layers.Dense(64, activation='relu'),
      tf.keras.layers.Dense(1)
  ])

model.compile(loss='mean_absolute_error', optimizer=tf.keras.optimizers.Adam(0.001))

I know I need to simply feed a tuple (x,y) into the model, but this is a dictionary of tuples. I tried to map it to separate out features and labels using:

dataset = dataset.map(lambda item: ((item['pt08s1'], item['nmhc'], item['c6h6']), item['co'])) 
#<MapDataset shapes: (((), (), ()), ()), types: ((tf.int32, tf.float32, tf.float32), tf.float32)>

and run

model.fit(dataset, epochs=2)

But I get an error:

ValueError: Layer my_model expects 1 input(s), but it received 3 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=() dtype=int32>, <tf.Tensor 'IteratorGetNext:1' shape=() dtype=float32>, <tf.Tensor 'IteratorGetNext:2' shape=() dtype=float32>]

I do not know how to figure it out actually. I appreciate all the help!

Answer 1

It is exactly similar to the problem posted here. I had just answered it yesterday.

The trick is to use named inputs layers and supply the input as a combined dict of x's and y's instead of tuple of x's and y's. For more details check the link above.