Search code examples
pythonstringfloating-pointscaleprecision

Determine precision and scale of particular number in Python


I have a variable in Python containing a floating point number (e.g. num = 24654.123), and I'd like to determine the number's precision and scale values (in the Oracle sense), so 123.45678 should give me (8,5), 12.76 should give me (4,2), etc.

I was first thinking about using the string representation (via str or repr), but those fail for large numbers (although I understand now it's the limitations of floating point representation that's the issue here):

>>> num = 1234567890.0987654321
>>> str(num) = 1234567890.1
>>> repr(num) = 1234567890.0987654

Edit:

Good points below. I should clarify. The number is already a float and is being pushed to a database via cx_Oracle. I'm trying to do the best I can in Python to handle floats that are too large for the corresponding database type short of executing the INSERT and handling Oracle errors (because I want to deal with the numbers a field, not a record, at a time). I guess map(len, repr(num).split('.')) is the closest I'll get to the precision and scale of the float?


Solution

  • Getting the number of digits to the left of the decimal point is easy:

    int(log10(x))+1
    

    The number of digits to the right of the decimal point is trickier, because of the inherent inaccuracy of floating point values. I'll need a few more minutes to figure that one out.

    Edit: Based on that principle, here's the complete code.

    import math
    
    def precision_and_scale(x):
        max_digits = 14
        int_part = int(abs(x))
        magnitude = 1 if int_part == 0 else int(math.log10(int_part)) + 1
        if magnitude >= max_digits:
            return (magnitude, 0)
        frac_part = abs(x) - int_part
        multiplier = 10 ** (max_digits - magnitude)
        frac_digits = multiplier + int(multiplier * frac_part + 0.5)
        while frac_digits % 10 == 0:
            frac_digits /= 10
        scale = int(math.log10(frac_digits))
        return (magnitude + scale, scale)