MATLAB : epipolar lines doesn't intersect with the points (fundamental matrix)

Question

I have a serious issue with my fundamental matrix computation and epipolar lines in matlab. You can see the result picture here : Matlab issue

As you can see, there is somehow a shift between the points and the lines. I should precise that I tried different methods than the following code : with the matlab function (epipolarline) and also with manually selected points between the two images rather than detection. I also computed the fundamental matrix manually (F_8 = transpose(inv(M_B))*St*inv(M_A);) but it didn't change anything.

It shouldn't be linked anyway to the epipolar lines not including their corresponding image point right ? I would be grateful if you could have a quick look and help me ! Here is my code :

    % 8 point algorithm %
    points_A = detectHarrisFeatures(rgb2gray(imgA));
    points_B = detectHarrisFeatures(rgb2gray(imgB));
    [featuresA, valid_pointsA] = extractFeatures(rgb2gray(imgA), points_A);
    [featuresB, valid_pointsB] = extractFeatures(rgb2gray(imgB), points_B);
    indexes = matchFeatures(featuresA,featuresB, 'MaxRatio', 0.65);
    matchedPointsA = valid_pointsA(indexes(:, 1), :);
    matchedPointsB = valid_pointsB(indexes(:, 2), :);

    F_8 = estimateFundamentalMatrix(matchedPointsA, matchedPointsB,'Method','Norm8Point');

    % Epipolar lines %
    figure()
    imgB = imread('asanB.jpg');
    imshow(imgB);
    hold on;

    for i = 1:size(image_points_B,1)
        line = F_8'*[matchedPointsB.Location(i,:),1]';

        points_x = [0,size(imgB,2)];
        points_y = [(-points_x(1)*line(1)-line(3))/line(2)...
            (-points_x(2)*line(1)-line(3))/line(2)];

        plot(matchedPointsB.Location(i,1),matchedPointsB.Location(i,2),'r.','MarkerSize',20)                         
        plot(points_x,points_y);
    end;

    hold off

Answer 1

This is perfectly normal. Estimation of the fundamental matrix is sensitive to noise, so the corresponding points rarely end up exactly on the epipolar line. Typically, you introduce some tolerance threshold to eliminate the bad matches that are too far from the corresponding epipolar lines.

You may be able to get a better result by starting out with more good matches. You may want to try a different interest point detector and/or a different descriptor. extractFeatures uses the FREAK descriptor with Harris corners by default. You can make it use a different descriptor by setting the 'Method' parameter.