i am using keras to train my sequential model with 3 layers and want to visualize gradient histograms in TensorBoard. For that there is the function "write_grads" in keras.callbacks.Tensorboard which should work if you define histogram_freq greater than 0 (keras docu). What i did:
### tensorboard call
callback_tb = keras.callbacks.TensorBoard(log_dir="logs/"+ name, write_graph = True, write_grads = True, histogram_freq=10 )
### some other callbacks
reduce_lr = keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=10, min_lr=0.001, verbose = 1)
early = keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=5, patience=10, verbose=1, mode='auto')
checkpointer = keras.callbacks.ModelCheckpoint(filepath='tmp/'+name+'.hdf5', verbose=1, save_best_only=True)
### model fit
model.fit(
X_train, y_train,
batch_size=1, nb_epoch=epochs, validation_split=0.05, verbose = 1,class_weight ={0: 1, 1: 0.5}, callbacks = [callback_tb, reduce_lr, early, checkpointer])
I have this model configuartion:
model = Sequential()
layers = [1, 100, 100, 100, 1]
model.add(GRU(
layers[1],
#batch_size = 209,
input_shape=(sequence_length, anzahl_features),
return_sequences=True))
model.add(Dropout(dropout_1))
model.add(LSTM(
layers[2],
#batch_size = 209,
return_sequences=True))
model.add(Dropout(dropout_2))
model.add(GRU(
layers[3],
#batch_size = 209,
return_sequences=False))
model.add(Dropout(dropout_3))
model.add(Dense(
layers[4]))
model.add(Activation('sigmoid'))
print(model.summary())
And the error message that i get is the following one:
TypeError: init() got an unexpected keyword argument 'write_grads'
Is there something wrong with my configuartion? Can i use this model and get the gradient histograms? Or are those histograms just available for a certain type of model?
You need to upgrade Keras to the latest release (2.0.5). Previous versions do not support the write_grads argument.
pip install keras --upgrade