Hyperparameter tuning with Hparam dashboard throwing error

Question

I am trying to optimize the best conditions for a sequential model I am building in keras.

I have recently come across Hparams dashboards which looks like a really nice way of doing this. However I am running into a problem at the stage of actually running the model to carry out the parameter optimization!

The code I am running (just to begin with taken directly from the tf page)

https://www.tensorflow.org/tensorboard/r2/hyperparameter_tuning_with_hparams

I have modified the code for Hparams on tf to my sequential model. For the purpose of practice I have removed a dropout layer (as I don't have any in my model) as well as the optimizer. For now I would like to see how my model is affected by changing nodes in layers. My code is as follows:

HP_NUM_UNITS = hp.HParam('num_units', hp.Discrete([16, 32]))


METRIC_ACCURACY = 'accuracy'

with tf.summary.create_file_writer('logs/hparam_tuning').as_default():
  hp.hparams_config(
    hparams=[HP_NUM_UNITS],
    metrics=[hp.Metric(METRIC_ACCURACY, display_name='Accuracy')],
  )

def train_test_model(hparams):
  model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(hparams[HP_NUM_UNITS], activation=tf.nn.relu),
    tf.keras.layers.Dense(24, activation=tf.nn.sigmoid),
  ])
  model.compile(
      optimizer='adam',
      loss='binary_crossentropy',
      metrics=['accuracy'],
  )

  model.fit(X_train.values, y_train, epochs=50) 
  _, accuracy = model.evaluate(X_test, y_test)
  return accuracy

def run(run_dir, hparams):
  with tf.summary.create_file_writer(run_dir).as_default():
    hp.hparams(hparams)  # record the values used in this trial
    accuracy = train_test_model(hparams)
    tf.summary.scalar(METRIC_ACCURACY, accuracy, step=1)

Up to this point, everything works fine! for the purpose of my first attempt, i have not changed much apart from removing dropout and optimizer plus applying my own model in the code. I require more units than 16 and 32 etc however this is just for the purpose of making a pipeline...

When I run the following code to execute the optimization, I get the error. the code is:

session_num = 0

for num_units in HP_NUM_UNITS.domain.values:
      hparams = { 
          HP_NUM_UNITS: num_units,
          }

      run_name = "run-%d" % session_num
      print('--- Starting trial: %s' % run_name)
      print({h.name: hparams[h] for h in hparams})
      run('logs/hparam_tuning/' + run_name, hparams)
      session_num += 1

This throws the error! the error is (which I don't quite understand):

ValueError: Cannot create an execution function which is comprised of elements from multiple graphs.

This error takes place following what looks like the first attempt at a model as for the first set of units (16) a model is fit. If i look at the traceback i get the progress report:

Epoch 1/50 140/140 [==============================] - 0s 3ms/sample - loss: 0.6847 - accuracy: 0.5723...... Epoch 50/50 140/140 [==============================] - 0s 206us/sample - loss: 0.2661 - accuracy: 0.8857

And after this is when I get the error( cannot create an execution function... etc)

I am unsure about how to fix this and any help would be much appreciated!

I am more than happy to provide any more detail/code!

Thank you!

Answer 1

I had the same error and I fixed it by turning my train and test values from pandas dataframe to numpy array. So just use X_train.values and so on so forth.

If this does just tell me at what line is the error exactly occurring at.