Search code examples
algorithmimage-processingcomputer-visionaffinetransformimage-registration

How to mathematically obtain camera trace path from video


I have an aerial video of some (approximately flat) view (no depth maps required).

I want to obtain the shape of the ground path the camera center passed upon.

The path does not have to be correct with correspondence to the worl'd north, or have any world coordinates. just have approximately the same shape, even if affine transformed.

I have already for each frame, the matrix that represents the affine transformation between that frame and the previous frame, at a good accuracy, meaning i have a matrix that for each pixel in a frame, gives the location of that pixel in the consecutive frame.

Given that the camera is moving, how can I obtain the SHAPE (over time) of the ground path of the spot on the ground that is in the center of the frames (=the trace)?

More generally, the question is actually only about knowing the relative center of the frame, for each frame, with correspondence to the first frame.

Any leads, as well as answers would benefit me a great deal.


Solution

  • Let's call the transformation between frame i and i+1 T. Let's look at a point P[i] = (Xi,Yi) in frame i. If we want to know were this point will be in frame i+1 we will just need to apply the transformation T on the point P[i] and get a new point P[i+1] = T(P).

    In order to get a path the camera did we start from frame 1 with the center point of the image P1 = (row/2,col/2) and calculate P2 = T(P1). The path the camera did between the frame 1 and 2 was approximately the line that connects between the center of frame 2 and P2. We can keep doing this for frame 2 by calculating P3 = T(P2) and so on... The problem is that the points are in in image coordinates and not world coordinate.

    To solve this we just decide that our "World coordinate system" PW1 = (0,0) is the center of the first frame. The center of the camera in the second frame in our new "World coordinate system" will be PW2 = ((row/2,col/2) - P2) - PW1. For the third frame the center will be PW3 = ((row/2,col/2) - P3) - PW2 and for the i-th frame PWi+1 = ((row/2,col/2) - Pi+1) - PWi. The idea is PW_new = movement_of_the_camera_between_frames - last_location.

    In the end you will have a set of points PW1,PW2,..,PWn which are the path the camera center did. Fit a spline or something and get the SHAPE of the path.