Why does this naive solution to catastrophic cancellation not work?

Question

I see on wikipedia that catastrophic cancellation is a phenomena where B~=A then A-B will have very high relative error compared to the true difference.

A quite naive solution occurred to me: why not just take: A-B~=(NA-NB)/N s.t. N>>1? This would make the 'true difference' much larger and should therefore decrease the relative error of approximating A-B by a lot right?

Answer 1

Consider a typical case where A and B are floating point numbers of the form M*(2^EXP). Catastrophic cancellation happens because M only has a limited number of bits, and M_A is approximately M_B so the high bits cancel. You only have a few significant bits left.

Now consider what happens is your solution, with N=16. That just performs the same calculation, except that the numbers now have the form M*(2^(EXP+4)). The problem is still M, not EXP.

You do have an additional problem, though, if EXP+4 overflows. Then the result would be INF-INF, which is NaN : Not a Number