I'm working on a project where I would like to reconstruct the 3D locations of feature points I've extracted from my camera images. The idea is to:
I've read in many papers that RANSAC is an algorithm that is used in reconstruction, with the end result being some kind of point cloud. I want to be able to do just that. However, I've ran into a few snags, and I hope you guys can help me out with these.
The first snag is that I do not really understand how I would be able to use RANSAC to perform this point correspondence. I understand the concept of RANSAC being a model-fitting tool, I just don't see how it could be used for doing correspondence solving.
The second snag is, assuming I have my correspondence information, how to get some kind of distance information between all these points. I've read that perspective projection could be used to solve this, and in turn one should try to estimate the Fundamental Matrix. Then do some math magic to be able to get the point cloud. Point is, I don't understand what the actual values in a Fundamental Matrix mean. I know it's gives a mathematical relation between the position of 2 cameras (or in my case, 2 frames in a video there the camera is moving), and that it exploits epipolar geometry. But besides this, I just don't have a clue what the Fundamental Matrix actually entails. How is this 3x3 matrix capturing the 6DOF of 1 camera with respect to another? Also I think the 'math magic' I referred to are some kind of matrix multiplications, but I haven't found any informational source to explain me what it does and what the formulation is.
Therefore, my question is: Could anyone of you point me into the right direction? I've been digging through the references of the papers I've read so far, but these also give me the "we solve this using the RANSAC algorithm"-line and I'm getting more the feeling I'm looking in the wrong direction. Is there some nice explanation of these things, perhaps in laymen's terms and/or with some illustrations? In short: where should I be looking or where can I find this elusive piece of information?
Thanks in advance, Xilconic
PS: Checked wikipedia, but it's not helping me much. Also listened to the 'Fundamental Matrix Song', and it's the same story.
Wrote my thesis on this, also using the RANSAC algorithm in my paper.
There is more to this topic than can be captured in a few paragraphs here. Consider getting the excellent book Multiple View Geometry.
RANSAC will find a model, in this case the fundamental matrix F, even in the presence of huge amount of outliers. In this case, some point-correspondance candidates are way off. This is an outlier. Basically you just keep fitting the F matrix from randomly drawn points. Eventually you find some set of points that together creates a consistent model. These are the inliers. They can now be used to estimate the model (F) more accurately.
There is an easy example in my paper with a line-fitting example to get you started and a easy-to-grasp explanation of RANSAC applied to the correspondance problem.
The most important thing about the F matrix is that it maps a point in one image to a line in the other:
Fx = l' where x is a point in one image and l' is a line in the other.
The F matrix has 9 elements but must have rank 2 and also the scale does not matter, thus it has only 7 degrees of freedom. There is no easy explanation for the elements of the F matrix.
Using a point correspondance x <-> x' and F the world 3d coordinate, X, of the depicted point can be extracted if you know the cameras internal parameters, like focal length.
Note that when using consecutive movie frames the camera usually moves very little and it might be hard to compute the fundamental matrix. It can be worked around though. I suggest looking into the works of Marc Pollefeys'