TensorFlow - Difference between tf.VariableScope and tf.variable_scope

Question

Have found two links on TensorFlow website referring to variable scope, one is tf.variable_scope which is the one most commonly used, and the other tf.VaribleScope.

As there's no example for the application of tf.VariableScope, just reading through the document, I couldn't distinguish whether there's a difference between the two. Tried to implement by replacing tf.variable_scope with tf.VariableScope but got the following error (which indicates there're some differences)

Traceback (most recent call last):
  File "/home/NER/window_model.py", line 105, in <module>
    model = NaiveNERModel(embeddings)
  File "/home/NER/window_model.py", line 64, in __init__
    pred = self.add_prediction_op(embed)
  File "/home/NER/window_model.py", line 82, in add_prediction_op
    with tf.VariableScope('Layer1', initializer=tf.contrib.layers.xavier_initializer()):
AttributeError: __enter__

Snippet of the original workable code

with tf.variable_scope('Layer1', initializer=tf.contrib.layers.xavier_initializer()):
        W = tf.get_variable("W", [self.dim * self.window_size, self.dim * self.window_size])
        b1 = tf.get_variable("b1", [self.dim * self.window_size])
        h = tf.nn.relu(tf.matmul(embed, W) + b1)

Answer 1

tf.VariableScope is an actual scope class that holds the name, initializer, regularizer, partitioner, and many other properties that are propagated to the variables defined in that scope. This class is more of a collection of properties and not a context manager, so you can't use it in with statement (that's what the error tells you).

Since the scope has to be first pushed on top of the stack (tensorflow internal class _VariableStore is responsible for this), then popped back from the stack, manual instantiation of tf.VariableScope is tedious and error-prone. That's where the context manager comes in.

tf.variable_scope is a context manager that makes it easier to work with variable scopes. As the documentation describes it:

A context manager for defining ops that creates variables (layers).

This context manager validates that the (optional) values are from the same graph, ensures that graph is the default graph, and pushes a name scope and a variable scope.

The actual work with variables is delegated to the tf.VariableScope object that is created under the hood.