How to transform a 3D bounding box from pointclouds to matched RGB frames in ROS?

Question

i am doing SLAM with depth camera realsense D455 on ROS (https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i) and I am able to create 3D point clouds of the environment. I am new in ROS and my goal is to bound a 3d box in a region of global pointcloud then transform the same box on RGB frames that are matched with that points(at the same coordinates) in the global point cloud. now i have RGB, depth, and point cloud topic and tf but since i am new in this field i do not know how can i find RGB frames that are matched with each point in the global point cloud? and how do the same operation from point cloud to RGB frames? i would be grateful if someone can help me with that

Answer 1

• Get 3D bounding box points.
• Transform them into camera's frame (z is depth).
• Generate your camera's calibration model (intrinsic and extrinsic parameters of the camera, focal length etc.) -> Check openCV model, Scaramuzza Model or another stereo camera calibration model
• Project 3D points to 2D pixels. (openCV has one projectPoints() function).