How can i have 2D bounding box on a sequence of RGBD frames from a 3D bounding box in point clouds?

Question

i have a 3d point clouds of my object by using Open3d reconstruction system ( makes point clouds by a sequence of RGBD frames) also I created a 3d bounding box on the object in point clouds my question is how can I have 2d bounding box on all of the RGB frames at the same coordinates of 3d bounding box? my idea Is to project 3d bb to 2d bb but as it is clear, the position of the object is different in each frame, so I do not know how can i use this approach? i appreciate any help or solution, thanks

Answer 1

• calculate points for the eight corners of your box
• transform those points from the world frame into your chosen camera frame
• project the points, apply lens distortion if needed.

OpenCV has functions for some of these operations and supports you with matrix math for the rest.

I would guess that Open3d gives you pose matrices for all the cameras. you use those to transform from the world coordinate frame to any camera's frame.