Search code examples
swiftdecimalnsdecimalnumber

NSDecimalNumber to UInt64 conversion issue

I have implementation of the function that is :

func number(for cof: UInt8, limit: UInt64) -> UInt64 {
    let decimalResult = (1 + Decimal(cof)/255) * Decimal(limit) / 1000
    return NSDecimalNumber(decimal: decimalResult).uint64Value
}

After usage of this function with 0 cof:

number(for: 0, gasLimit: 21000) // Result is 21

But when I call function with different cof value.

number(for: 128, gasLimit: 21000) // Result is 0

Afer debug for 128 cof value.

I found out that 

 let decimalResult = (1 + Decimal(gasPriceCoef)/255) * Decimal(gasLimit) / 1000 // Result is 31.5411764705882352941176470588235294116

And problem is that when I convert Decimal value to UInt64 I receive 0

Solution

  • uint64Value is a property of NSNumber, and it is not well-documented when applied to NSDecimalNumber.

    But as far as I tested, when mantissa part exceeds the range of UInt64, it returns 0. For example, 31.5411764705882352941176470588235294116 is represented as 315411764705882352941176470588235294116×10^(-38) in NSDecimalNumber, and 315411764705882352941176470588235294116 is bigger than UInt64.max. (*1)

    To avoid this behavior (*2), you can round the decimal value before converting it to UInt64.

    func number(for cof: UInt8, limit: UInt64) -> UInt64 {
    var decimalResult = (1 + Decimal(cof)/255) * Decimal(limit) / 1000
    var roundedResult = Decimal()
    NSDecimalRound(&roundedResult, &decimalResult, 0, NSDecimalNumber.RoundingMode.plain)
    return NSDecimalNumber(decimal:  roundedResult).uint64Value
}
print(number(for: 128, limit: 21000)) //->32

    (*1) Actual behavior seems a little bit more complicated, see the Shorter examples in Martin R's comment above.

    (*2) This behavior definitely is a bug, once marked as RESOLVED. See also another comment of Martin R's above.