Search code examples
graphics3dcomputational-geometryprojection

Why does the projection of an image over 3d points show this distortion?

I have a question regarding the projection of an image over a set of 3D points. The image is given to me as a JPG, together with position and attitude information of the camera relative to a cartesian coordinate system (Xc,Yc,Zc and yaw, pitch, roll), as well as the horizontal and vertical field of view (in degrees).

Points are given using solely their 3d position in the same coordinate system (Xp,Yp,Zp).

In my coordinate system, Z is up. To project the image onto the points, I

  • compute the vector from camera to each point

    Vector3 c2p = (Xp,Yp,Zp)-(Xc,Yc,Zc);

  • rotate c2p according to my camera's attitude (quaternion):

    Vector3 c2pCamFrame = getCamQuaternion().conjugate().rotate(c2p);

  • compute azimuth and elevation from the camera's "center ray" to the point:

    float azimuth = atan2(c2pCamFrame.x(),c2pCamFrame.y()));
float elevation = atan2(c2pCamFrame.z(),sqrt(pow(c2pCamFrame.x(),2)+pow(c2pCamFrame.y(),2)));

  • if azimuth and elevation are within the field of view, I assign the color of the corresponding pixel to the point.

This works almost perfectly, and the "almost" motivates my question. Let me show you: Diagram

I cannot figure out why the elevation of the projection is distorted. In the bottom right of the image, you can see that points outside the frustum (exceeding the elevation) actually become colored - and this distortion is null at an azimuth of 0 degrees and peaks at the left and right edges of the image, creating the pillow distortion.

Why does this distortion appear? I'd love to understand this problem both in geometrical as well as mathematical terms. Thank you!

Solution

  • The field of view angles are only valid on the principal axes. But you can do it the other way around. I.e. calculate the x/y bounds from the angles:

    maxX = tan(horizontal_fov / 2)
maxY = tan(vertical_fov / 2)

    And check

    if(abs(c2pCamFrame.x() / c2pCamFrame.z()) <= maxX
     && abs(c2pCamFrame.y() / c2pCamFrame.z()) <= maxY)

    Additionally, you might want to check if the points are in front of the camera:

         ... && c2pCamFrame.z() > 0

    This assumes a left-handed coordinate system.