Search code examples
pythonopencvimage-processingcomputer-visiontriangulation

3D Object Position Estimation Relative to Fix Points Using Two Non-static Cameras

I am trying to estimate the motion of an object (marker M) in 3D with the help of 2 non-static cameras (See attached Figure). Apart from the moving marker, M , there are 4 static co-planar markers as well in the scene (m1, m2, m3 and m4). Absolute distance (in centimeters) between static markers is known.

enter image description here

The two cameras can move around, but are fixed relative to each other. The cameras take images of the scene simultaneously. I can detect all the markers (static as well as mobile) in 2D images from both the cameras.

The goal is to calculate the motion of M in 3D across frames. One way to do this is to assume m1 as origin (0,0,0) of the world coordinate system, and find 3D position of M (x,y,z) across two frames, and then find the distance between these two positions.

Hence, the problem resolves to calculating position of M wrt to m1 in 3D, across frames. This means that ego-motion (motion due to camera) across frames should have no effect on the position of M as it is relative to m1, which is origin.

I am looking for someone to guide me towards the right direction to solve this problem using OpenCV.

Solution

  • What I would try is the following:

    1. Estimate the pose of both cameras. Since you have the image positions of the markers, and their 3d position, you can do this using PnP.
    2. Triangulate the position of M using the estimated camera positions.

    The coordinate frame will be decided by the 3d positions of the markers, so m1 will be your origin.

    This requires that

    1. At least three markers are visible in each camera image
    2. M is visible in both camera images
    3. Your cameras are calibrated

    You should be able to find the OpenCV functions that perform the two steps above.