Why do i get double info string only when i use pretrained slim model?

Question

I start training a faster_rcnn_inception_v2 with inception_v2_imagenet_2016_08_28 pretrained model from slim.

  fine_tune_checkpoint: "./pretrained_models/inception_v2_imagenet_2016_08_28/inception_v2.ckpt"
  from_detection_checkpoint: false

I get a warning about missing parameter (gamma). Than I get all the info doubled.

INFO:tensorflow:global step 435766: loss = 0.7736 (0.27 sec/step)  
INFO:tensorflow:global step 435766: loss = 0.7736 (0.27 sec/step)

Why? Is there a solution? Thanks.

Answer 1

I found the solution to this problem!

The bug is in the function get_variables_available_in_checkpoint in https://github.com/tensorflow/models/blob/master/research/object_detection/utils/variables_helper.py:

def get_variables_available_in_checkpoint(variables, checkpoint_path):
    """Returns the subset of variables available in the checkpoint.

    Inspects given checkpoint and returns the subset of variables that are
    available in it.

    TODO: force input and output to be a dictionary.

    Args:
      variables: a list or dictionary of variables to find in checkpoint.
      checkpoint_path: path to the checkpoint to restore variables from.

    Returns:
      A list or dictionary of variables.
    Raises:
      ValueError: if `variables` is not a list or dict.
    """
    if isinstance(variables, list):
        variable_names_map = {variable.op.name: variable for variable in variables}
    elif isinstance(variables, dict):
        variable_names_map = variables
    else:
        raise ValueError('`variables` is expected to be a list or dict.')
    ckpt_reader = tf.train.NewCheckpointReader(checkpoint_path)
    ckpt_vars = ckpt_reader.get_variable_to_shape_map().keys()
    vars_in_ckpt = {}
    for variable_name, variable in sorted(variable_names_map.items()):
        if variable_name in ckpt_vars:
            vars_in_ckpt[variable_name] = variable
        else:
            logging.warning('Variable [%s] not available in checkpoint',
                            variable_name)
    if isinstance(variables, list):
        return vars_in_ckpt.values()
    return vars_in_ckpt

I commented this part and the info during training are displayed only once.

    # else:
    #     logging.warning('Variable [%s] not available in checkpoint',
    #                     variable_name)