'ValueError: Image with id {} already added' When Run Evaluation on OpenImage Dataset

Question

I am using Tensorflow Object Detection API to evaluate the SSD_mobilenet model on OpenImageV5 dataset. I used the pretained model provided in their model zoo. For the pipeline config, I am using the default pipeline (ssd_mobilenet_v2_oid_v4) available here.

When I run the evaluation using following command:

python object_detection/model_main.py --alsologtostderr --run_once --checkpoint_dir=/Path/TO/ssd_mobilenet_v2_oid_v4_2018_12_12/ --model_dir=eval/ --pipeline_config_path=PATH/TO/ssd_mobilenet_v2_oid_v4.config

I get this error:

2019-05-28 12:26:09.658946: W tensorflow/core/framework/op_kernel.cc:1306] Invalid argument: ValueError: Image with id 1407414044 already added.
Traceback (most recent call last):

  File "/bigSSD/tensorflow_model/v_env_tf_1.8/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 157, in __call__
    ret = func(*args)

  File "/bigSSD/tensorflow_model/models/research/object_detection/utils/object_detection_evaluation.py", line 432, in update_op
    self.add_single_ground_truth_image_info(image_id, single_example_dict)

  File "/bigSSD/tensorflow_model/models/research/object_detection/utils/object_detection_evaluation.py", line 600, in add_single_ground_truth_image_info
    raise ValueError('Image with id {} already added.'.format(image_id))

ValueError: Image with id 1407414044 already added.

I changed sample_1_of_n_examples from 10 to 100 and 1000 but still got the same error.

Answer 1

for faster_rcnn_inception_resnet_v2_atrous_oid_v4.config I changed

eval_config 
{
  metrics_set: "open_images_V2_detection_metrics"
}

to

eval_config {
  metrics_set: "open_images_V2_detection_metrics"
  num_examples: 383 #correspond to my test images
}

then run the evaluation with eval.py using the following command:

python eval.py --logtostderr --checkpoint_dir=path/to/checkpoint --eval_dir=path/to/eval_dir --pipeline_config_path=path/to/pretrained_model.config

and my error solved.