ValueError: The `batch_size` argument must not be specified for the given input type

Question

I have created dummy data for an autoencoder model using tf.data.dataset. I am using tensorflow-1.15.x, keras 2.3.1 and numpy 1.19.5 for running on my GPU.

N = 5000
H = 128
W = 128
C = 2

train_data = np.random.randn(N,H,W,C)

train_data = tf.convert_to_tensor(train_data)

test_data = np.random.randn(500,H,W,C)

test_data = tf.convert_to_tensor(test_data)

batch_size = 1

train_dataset = tf.data.Dataset.from_tensor_slices((train_data, train_data))
train_dataset = train_dataset.batch(batch_size, drop_remainder=True)

test_dataset = tf.data.Dataset.from_tensor_slices((test_data, test_data))
test_dataset = test_dataset.batch(batch_size, drop_remainder=True)

But while fitting this data on the model, I get the following error -

epochs = 5

rms = RMSprop(learning_rate=0.00002,
              rho=0.9,
              epsilon=1e-07)

model.compile(loss='mean_squared_error', optimizer=rms, metrics=['mean_squared_error'])

callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=3)

logs = "logs/"

tboard_callback = tf.keras.callbacks.TensorBoard(log_dir = logs,
                                                 histogram_freq = 1)

history = model.fit(train_dataset, epochs=epochs, batch_size=batch_size, callbacks=[callback, tboard_callback],
                    validation_data=test_dataset)


---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-25-47134a802115> in <module>()
     15 
     16 history = model.fit(train_dataset, epochs=epochs, batch_size=batch_size, callbacks=[callback, tboard_callback],
---> 17                     validation_data=test_dataset)

2 frames
/tensorflow-1.15.2/python3.7/tensorflow_core/python/keras/engine/training.py in _validate_or_infer_batch_size(self, batch_size, steps, x)
   1813             'The `batch_size` argument must not be specified for the given '
   1814             'input type. Received input: {}, batch_size: {}'.format(
-> 1815                 x, batch_size))
   1816       return
   1817 

ValueError: The `batch_size` argument must not be specified for the given input type. Received input: <DatasetV1Adapter shapes: ((1, 128, 128, 2), (1, 128, 128, 2)), types: (tf.float64, tf.float64)>, batch_size: 1

Answers to other questions on this error suggest batching both train and validation datasets, but I have batched both already. What could be causing this error? Thanks.

Answer 1

Remove the batch_size in model.fit()

history = model.fit(train_dataset, epochs=epochs, callbacks=[callback, tboard_callback],validation_data=test_dataset)