People familiar with Homographies will know that you have to normalize it dividing by any of the matrix components in order to keep homogeneous coordinates. An homography is a 3x3 matrix and it is usually normalized dividing by the element at (3,3).
The problem comes when that value is very small (for example 0.0000008) and divides a value that is supossed to be zero (0.0000007). The resulting value is almost 0.875 when it was supossed to be zero and the resulting projection has no sense.
I would like to know which is the common way to solve this. I use C++ and floating point arithmetic.
So, if i understand the question:
0/0.000000001 = 0 = CORRECT
and:
0.000000001/0.000000001 ~ 1 INCORRECT
I will define a function to check the error, with a parameter sigma.
If value < sigma = 0.001, assume its zero, and return 0, else, return value.
So, it will work always with value over the sigma error, and 0 if not.