Find real world coordinate of a point from calibrated stereo cameras with fundamental and essential matrix

Question

I have a stereoscopic camera set up with two webcams that I am using with Matlab. I calibrate the cameras, and get the stereoParams.

Then, I want a user to be able to select a point in a picture, and get the real world point location in the image. I know for this I need the baseline, the focal length, and the pixel disparity. I have the pixel disparity, but how do I get the baseline and focal length? Can baseline be calculated from the stereoParams?

Answer 1

I am not familiar with the Matlab stereo camera calibration functions, but in general, once you calibrate each camera, and find the fundamental matrix, you should be able to do the following:

1. Set one of the images as reference and rectify the other image so that disparity search proceeds along horizontal lines in the image
2. From the pixel disparity, you can calculate the real world depth by the relation z = fB/d, where f is the focal length, B is the baseline, and d is the disparity. It is very important to mind the units! if d is in pixels, then f must also be in pixels if you want z to be in the units of the baseline (e.g. centimeters)
3. The baseline is the distance between the optical centers of the cameras. It should be available from the matlab stereoParameters.translationofCamera2
4. The focal length is an intrinsic parameter of each camera. I assumed equal focal lengths above, but with webcams, this is not guaranteed. You should be able to extract the focal length from the matlab cameraParameters.IntrinsicMatrix. The focal length is related to the alpha parameters in the intrinsic matrix (see this Wikipedia entry for explanations)