keras with tensorflow runs fine, until I add callbacks

Question

I'm running a model using Keras and TensorFlow backend. Everything works perfect:

model = Sequential()
model.add(Dense(dim, input_dim=dim, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(1, activation='linear'))

model.compile(loss='mse', optimizer='Adam', metrics=['mae'])

history = model.fit(X, Y, epochs=12, 
                    batch_size=100, 
                    validation_split=0.2, 
                    shuffle=True, 
                    verbose=2)

But as soon as I include logger and callbacks so I can log for tensorboard, I get

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_layer_input_2' with dtype float and shape [?,1329]...

Here's my code: (and actually, it worked 1 time, the very first time, then ecer since been getting that error)

model = Sequential()
model.add(Dense(dim, input_dim=dim, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(1, activation='linear'))

model.compile(loss='mse', optimizer='Adam', metrics=['mae'])

logger = keras.callbacks.TensorBoard(log_dir='/tf_logs',
                                     write_graph=True,
                                     histogram_freq=1)

history = model.fit(X, Y, 
                    epochs=12,
                    batch_size=100,
                    validation_split=0.2,
                    shuffle=True,
                    verbose=2,
                    callbacks=[logger])

Answer 1

A tensorboard callback uses tf.summary.merge_all function in order to collect all tensors for histogram computations. Because of that - your summary is collecting tensors from previous models not cleared from previous model runs. In order to clear these previous models try:

from keras import backend as K

K.clear_session()

model = Sequential()
model.add(Dense(dim, input_dim=dim, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(1, activation='linear'))

model.compile(loss='mse', optimizer='Adam', metrics=['mae'])

logger = keras.callbacks.TensorBoard(log_dir='/tf_logs',
                                 write_graph=True,
                                 histogram_freq=1)

history = model.fit(X, Y, 
                epochs=12,
                batch_size=100,
                validation_split=0.2,
                shuffle=True,
                verbose=2,
                callbacks=[logger])