Search code examples
.netvb.nettype-conversion

Long.MaxValue conversion from Single to Long causing Overflow exception

The value of Long.MaxValue is 9,223,372,036,854,775,807. When cast to a single precision value, becomes 9.223372E18. However the cast back to a Long throws a System.OverflowException. This despite the fact that the value in the Single is actually less than the possible maximum value of the long. Experimentation in the watch window shows that in fact the value

8.070450532247929344e18

is the largest double precision value that will safely ctype to a Long. Try this in the watch window:

CType(8.070450532247929344000001e18, Long)

further

8.070450807125836288e18

is the largest double precision value that after conversion to single precision, will safely convert to a Long, using this statement in the watch window:

CType(CType(8.0704508071258362880000001e18, Single), Long)

This seems like very strange behaviour.

Why?

Edit: Further Experimentation

The above experimentation is using the watch window and CTypes. Another experiment yields another answer:

Dim d As Long = 200000000000
Dim delta As Long = d / 2
Dim c As Long
Dim a As Single
Dim b As Long
While delta > 0
    c = Long.MaxValue - d
    a = c
    Try
        b = a
        delta = delta \ 2

        d = d - delta
    Catch ex As Exception
        d = d + delta

    End Try
End While

The above code yields the largest Long that can be cast back and forth is:

9,223,371,761,976,868,860

or

Long.MaxValue - 274,877,906,947
Solution

  • When cast to a single precision value, Long.MaxValue becomes 9.223372E18, which is less than the possible maximum value of the Long.

    Although 9.223372E18 is the default string representation of the result, this is misleading. The exact result is not 9.223372 × 1018 but rather 263 = 9,223,372,036,854,775,808, which is the closest possible Single to the original value of 263 − 1. (The difference of 1 is due to the inability of Single to precisely represent the original value.)

    Because 263 exceeds Long.MaxValue, attempting to cast it back to a Long throws OverflowException.

    What is the largest Long that can be represented exactly by a Single?

    We want to find the maximum possible value of L = 2e × s < 263, where the integer exponent e satisfies −126 ≤ e ≤ 127, and the significand s = (1.b22b21...b0)2 consists of the binary digit 1 followed by 23 fractional binary digits.

    • If e ≥ 63, then L ≥ 263, which is too large. So e = 62.
    • The maximum possible s is (1.11...1)2 = 2 − 2−23.

    Thus L = 262 × (2 − 2−23) = 263 − 239 = 9,223,371,487,098,961,920.

    What is the largest Long that can be converted to a Single and back to a Long without overflowing?

    The answer M is the midpoint of L (from above) and Long.MaxValue:

    M = (L + 263 − 1) \ 2 = 263 − 238 − 1 = 9,223,371,761,976,868,863.

    When you convert any Long between L and M to a Single, you get L:

    Dim a As Long = 9223371761976868863
Dim b As Single = a
Dim c As Long = CType(b, Long)
Console.WriteLine(c) ' Output: 9223371487098961920

    When you convert any Long between M + 1 and Long.MaxValue to a Single, you get 263, which exceeds Long.MaxValue.

    Why does CType(8.07045053224793E+18, Long) fail to compile?

    Good question. A Double should be convertible to a Long at compile time if its value is within the range of the Long data type (approximately ±9.223E+18). 8.07045053224793E+18 is less than Long.MaxValue, and in fact the conversion succeeds at run time:

    ' This code generates error BC30439: Constant expression not representable in type 'Long'.
Const a = 8.07045053224793E+18
Const b As Long = CType(a, Long)

' But the identical code with Dim instead of Const compiles successfully.
Dim a = 8.07045053224793E+18
Dim b As Long = CType(a, Long)
Console.WriteLine(b) ' Output: 8070450532247929856

    Therefore, this is clearly a bug in VB.NET’s compile-time conversion logic. Indeed, I traced the bug to the following decade-old block of code within the compiler, which is supposed to check whether the sourceValue is between &H7000000000000000L and &H8000000000000000UL, and if so, return False to indicate "no overflow":

    If sourceValue > &H7000000000000000L Then
    Dim temporary As Double = (sourceValue - &H7000000000000000L)

    If temporary < &H7000000000000000L AndAlso UncheckedCLng(temporary) > &H1000000000000000L Then
        Return False
    End If
Else

    But the developer incorrectly used > instead of < in the comparison UncheckedCLng(temporary) > &H1000000000000000L. I’ve submitted a pull request to get this corrected.

    The usage of the somewhat arbitrary constant &H7000000000000000L in this code explains the strange behavior you observed: converted to a Double, this constant equals 8.0704505322479288E+18.