Detectron2 Panoptic FPN Model Partial Execution - TypeError: 'NoneType' object is not iterable

Question

I am trying to extract the pre-output feature-map for the Panoptic Output of Detectron2 ResNet50 - based FPN model.

Hence, In order to get partial model outputs, I am following the official Detectron2 Modeling Documentation to Partially Execute Models.

Please find the code below:

# Setting the model Configurations
cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file(
    "COCO-PanopticSegmentation/panoptic_fpn_R_50_1x.yaml")
)
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(
    "COCO-PanopticSegmentation/panoptic_fpn_R_50_1x.yaml"
)

# Build the model from config
model = build_model(cfg)

# Loading an image just for testing
im = cv2.imread("./detectron/input.jpg")
im = torch.from_numpy(im).cuda().permute(2, 0, 1).unsqueeze(0).float()

# Extracting Features - This part works fine
features = model.backbone(im)

# Extracting the proposals - Throws Error
proposals = model.proposal_generator(im, features)

Please find the error thrown by the above step shown below:

TypeError                                 Traceback (most recent call last)
<ipython-input-17-dd53471cf2d2> in <module>
----> 1 proposals = model.proposal_generator(im, features)

~/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

/media/Data/Documents/Python-Codes/Freelance/detectron2/detectron2/modeling/proposal_generator/rpn.py in forward(self, images, features, gt_instances)
    409 
    410         if self.training:
--> 411             gt_labels, gt_boxes = self.label_and_sample_anchors(anchors, gt_instances)
    412             losses = self.losses(
    413                 anchors, pred_objectness_logits, gt_labels, pred_anchor_deltas, gt_boxes

~/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     13         def decorate_context(*args, **kwargs):
     14             with self:
---> 15                 return func(*args, **kwargs)
     16         return decorate_context
     17 

/media/Data/Documents/Python-Codes/Freelance/detectron2/detectron2/modeling/proposal_generator/rpn.py in label_and_sample_anchors(self, anchors, gt_instances)
    272         anchors = Boxes.cat(anchors)
    273 
--> 274         gt_boxes = [x.gt_boxes for x in gt_instances]
    275         image_sizes = [x.image_size for x in gt_instances]
    276         del gt_instances

TypeError: 'NoneType' object is not iterable

Please let me know what am I doing wrong, and how to fix it. An explanation regarding why did this error come up would also be very helpful.

Please let me know if any other information is required.

Answer 1

With a bit more digging, I solved the issue. There were a couple of problems in the above code:

1. I did not set the model to eval mode first - model.eval(). The model needs to be set to eval fist.
2. The mode.proposal_generator() expects inputs in the form of ImageList object, details regarding which can be found here.

Performing the above two steps solved the issue.