Search code examples
c#unity-game-enginevectormath

Vector3.SqrMagnitude seems inaccurate

I have a function to return the total length of a NavMesh path, using Vector3.SqrMagnitude to avoid the overhead of the Sqrt calculation in Vector3.Distance:

float DistanceAlongPath(NavMeshPath path)
{
    float sum = 0;

    for (int x = 0; x < path.corners.Length - 1; x++)
    {
        sum += Vector3.SqrMagnitude(path.corners[x + 1] - path.corners[x]);
    }

    return sum;
}

When the path only has two points the calculation seems fine, however at 3 points or more it always returns a smaller value for a long line than it does for a short line of only 2 points.

The distance shown in the first image is 3848, in the second it's 3419 despite being a much longer path.

enter image description here enter image description here

Vector3.Distance works correctly with any number of points.

Am I missing something?

Solution

  • SqrMagnitude gives only an approximation of length. While values of SqrMagnitude can be compared to each other for relative length comparison, they do not give a true length, so you can't just combine them.

    The root of your problem here is in the math, specifically in the order in which addition and multiplication are handled:

    (5+5+5)^2 != (5^2 + 5^2 + 5^2)

    For example, if you have 3 line segments, each of length 5, each will have SqrMagnitude of 25. Add them and you get 115.

    Now consider a single line segment with length 15. SqrMagnitude is 225.

    In either case, if you apply the square root operation to get a true length, you get the same result. 3*Sqrt(25) == 1*Sqrt(225)