Search code examples
tensorflowobject-detection-apifaster-rcnntensorflow-model-garden

Visualize proposal regions from RPN head in Faster R-CNN with Tensorflow Object Detection API

I'm trying debug my trained Faster R-CNN model using Tensorflow Object Detection API and I want to visualize the proposal regions of RPN on an image. Can anyone tell me how to do it? I found a post here but it hasn't been answered. I tried to export the model using exporter_main_v2.py with only the RPN head as said here and this is the massage when I deleted the second_stage.

Traceback (most recent call last):
  File "exporter_main_v2.py", line 165, in <module>
    app.run(main)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 312, in run
    _run_main(main, args)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "exporter_main_v2.py", line 158, in main
    exporter_lib_v2.export_inference_graph(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py", line 245, in export_inference_graph
    detection_model = INPUT_BUILDER_UTIL_MAP['model_build'](
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\builders\model_builder.py", line 1226, in build
    return build_func(getattr(model_config, meta_architecture), is_training,
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\builders\model_builder.py", line 665, in _build_faster_rcnn_model
    second_stage_box_predictor = box_predictor_builder.build_keras(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\builders\box_predictor_builder.py", line 991, in build_keras
    raise ValueError(
ValueError: Unknown box predictor for Keras: None

I tried again to export the model without deleting the second_stage. And this is the message I got

INFO:tensorflow:depth of additional conv before box predictor: 0
I0802 20:55:13.930429  1996 convolutional_keras_box_predictor.py:153] depth of additional conv before box predictor: 0
Traceback (most recent call last):
  File "exporter_main_v2.py", line 165, in <module>
    app.run(main)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 312, in run
    _run_main(main, args)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\absl\app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "exporter_main_v2.py", line 158, in main
    exporter_lib_v2.export_inference_graph(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py", line 271, in export_inference_graph
    concrete_function = detection_module.__call__.get_concrete_function()
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 1299, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 1205, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 725, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\function.py", line 2969, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\function.py", line 3361, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\function.py", line 3196, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\framework\func_graph.py", line 990, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\eager\def_function.py", line 634, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "E:\Anaconda\envs\TFOD\lib\site-packages\tensorflow\python\framework\func_graph.py", line 977, in wrapper
    raise e.ag_error_metadata.to_exception(e)
tensorflow.python.autograph.pyct.error_utils.KeyError: in user code:

    E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py:163 call_func  *
        return self._run_inference_on_images(images, true_shapes, **kwargs)
    E:\Anaconda\envs\TFOD\lib\site-packages\object_detection\exporter_lib_v2.py:129 _run_inference_on_images  *
        detections[classes_field] = (

    KeyError: 'detection_classes'
Solution

  • Found the solution!
    In the config file add number_of_stages: 1
    Instead of using exporter_main_v2.pyI write code that builds the model from the checkpoint file

    # Load pipeline config and build a detection model
configs = config_util.get_configs_from_pipeline_file(path_to_config)
model_config = configs['model']
detection_model = model_builder.build(model_config=model_config, is_training=False)

# Restore checkpoint
ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
ckpt.restore(os.path.join(path_to_ckpt, 'ckpt-0')).expect_partial()

    Then I feed the image I need to inspect to the model, then I use object_detection.utils.visualization_utils.visualize_boxes_and_labels_on_image_array to inspect the boxes