Tensorflow OD API - pipeline.config changes after exporting trained model

Question

I'm using Tensorflow OD API 1.13 to do transfer learning of pre-trained ssd_mobilent_v2_coco from the model zoo. After finishing training, I export the frozen graph using the command:

python object_detection\export_inference_graph.py ^
--input_type image_tensor ^
--pipeline_config_path C:\TF_train\models\model\pipeline.config ^
--trained_checkpoint_prefix C:\TF_train\models\model\model.ckpt-7200 ^
--output_directory C:\export\ssd_v2_7200

In the output folder, there are model files and a pipeline.config file. But when I open it and compare to the original one (used to train the model) I see that some fields were changed.

For example, here's the feature extractor section from the original pipeline.config:

feature_extractor {
  type: "ssd_mobilenet_v2"
  depth_multiplier: 1.0
  min_depth: 16
  conv_hyperparams {
    regularizer {
      l2_regularizer {
        weight: 3.99999989895e-05
      }
    }
    initializer {
      truncated_normal_initializer {
        mean: 0.0
        stddev: 0.0299999993294
      }
    }
    activation: RELU_6
    batch_norm {
      decay: 0.999700009823
      center: true
      scale: true
      epsilon: 0.0010000000475
      train: true
    }
  }
  use_depthwise: true
}

and from exported pipeline.config:

feature_extractor {
   type: "ssd_mobilenet_v2"
   depth_multiplier: 1.0
   min_depth: 16
   conv_hyperparams {
     regularizer {
       l2_regularizer {
         weight: 3.9999998989515007e-05
       }
     }
     initializer {
       truncated_normal_initializer {
         mean: 0.0
         stddev: 0.029999999329447746
       }
     }
     activation: RELU_6
     batch_norm {
       decay: 0.9997000098228455
       center: true
       scale: true
       epsilon: 0.0010000000474974513
       train: true
     }
   }
   use_depthwise: true
 }

Notice, how precision changed for l2_regularizer weight, stddev, decay and epsilon. Is this expected? Why is this happening?

Answer 1

Turns out it's a protobuf issue:

We have a known issue for float type precision if it is using cpp extension:

Python does not have C-style float , it only has a C-style double. Thus pure python is using double precision for both float and double field, cpp extension is using float precision for float field.

I checked the protobuf active implementation like so:

python -c "from google.protobuf.internal import api_implementation; print(api_implementation.Type())"

The output was cpp. So I added the environment variable:

PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

And now it works as expected.