Tensorflow forward pass with dropout

Question

I am trying to use dropout to get error estimates for a neural network. This involves running several forward passes of my network during not only training but also testing, with dropout activated. Dropout layers seem only activated while training but not testing. Can this be done in Tensorflow by just calling some functions or modifying some parameters?

Answer 1

Yes, the easiest way is to use tf.layers.dropout that has a training argument, which can be tensor that you can define by true or false in a any particular session run:

mode = tf.placeholder(tf.string, name='mode')
training = tf.equal(mode, 'train')

...

layer = tf.layers.dropout(layer, rate=0.5, training=training)

...

with tf.Session() as sess:
  sess.run(..., feed_dict={mode: 'train'})  # This turns on the dropout

  sess.run(..., feed_dict={mode: 'test'})   # This turns off the dropout