Search code examples
matlabimage-processingcomputer-visionobject-recognition

Lukas Kanade optical flow: Understanding the math


I found a Matlab implementation of the LKT algorithm here and it is based on the brightness constancy equation.

The algorithm calculates the Image gradients in x and y direction by convolving the image with appropriate 2x2 horizontal and vertical edge gradient operators.

The brightness constancy equation in the classic literature has on its right hand side the difference between two successive frames.

However, in the implementation referred to by the aforementioned link, the right hand side is the difference of convolution.

It_m = conv2(im1,[1,1;1,1]) + conv2(im2,[-1,-1;-1,-1]);

Why couldn't It_m be simply calculated as:

it_m = im1 - im2;     

Solution

  • (Comments converted to an answer.)

    In theory, there is nothing wrong with taking a pixel-wise difference:

    Im_t = im1-im2;
    

    to compute the time derivative. Using a spatial smoother when computing the time derivative mitigates the effect of noise.

    Moreover, looking at the way that code computes spatial (x and y) derivatives:

    Ix_m = conv2(im1,[-1 1; -1 1], 'valid');

    computing the time derivate with a similar kernel and the valid option ensures the matrices It_x, It_y and Im_t have compatible sizes.