Search code examples
postgresqltypesnumeric

postgresql: datatype numeric with limited digits


I am looking for numeric datatype with limited digits (before and after the decimal point)

The function kills only digits after the decimal point. (PG version >= 13)

create function num_flex( v numeric, d int) returns numeric as 
$$
  select case when v=0 then 0 
              when v  < 1 and  v > -1  then trim_scale(round(v, d - 1 ) )
              else trim_scale(round(v, d - 1 - least(log(abs(v))::int,d-1) ) ) end;
$$
language sql ;

For testing:

select num_flex( 0, 6) 
union all
select num_flex( 1.22000, 6) 
union all
select num_flex( (-0.000000123456789*10^x)::numeric,6) 
from generate_series(1,15,3) t(x) 
union all
select num_flex( (0.0000123456789*10^x)::numeric,6) 
from generate_series(1,15,3) t(x) ;

It runs, but have someone a better idea or find a bug (a situation, that is not implemented)?

The next step is to integrate this in PG, so that I can write

select 12.123456789::num_flex6 ; 
select 12.123456789::num_flex7 ;

for a num_flex datatype with 6 or 7 digits. with types from num_flex2 to num_flex9. Is this possible?


Solution

  • There are a few problems with your function:

    1. Accepting negative digit counts (parameter d). num_flex(1234,-2) returns 1200 - you specified you want the function to only kill digits after decimal point, so 1234 would be expected.
    2. Incorrect results between -1 and 1. num_flex(0.123,3) returns 0.12 instead of 0.123. I guess this might also be desired effect if you do want to count 0 to the left of decimal point. Normally, that 0 is ignored when a number's precision and scale are considered.
    3. Your counting of digits to the left of decimal point is incorrect due to how ::int rounding works. log(abs(11))::int is 1 but log(abs(51))::int is 2. ceil(log(abs(v)))::int returns 2 in both cases, while keeping int type to still work as 2nd parameter in round().
    create or replace function num_flex( 
        input_number numeric,
        digit_count int,
        is_counting_unit_zero boolean default false)
    returns numeric as 
    $$
      select trim_scale(
                case 
                    when input_number=0 
                        then 0 
                    when digit_count<=0 --avoids negative rounding
                        then round(input_number,0)
                    when (input_number between -1 and 1) and is_counting_unit_zero
                        then round(input_number,digit_count-1)
                    when (input_number between -1 and 1)
                        then round(input_number,digit_count)
                    else 
                        round(  input_number, 
                                greatest( --avoids negative rounding
                                    digit_count - (ceil(log(abs(input_number))))::int,
                                    0)
                        )
                end
              );
    $$
    language sql;
    

    Here's a test

    select *,"result"="should_be"::numeric as "is_correct" from 
    (values
        ('num_flex(0.1234      ,4)',num_flex(0.1234      ,4), '0.1234'),
        ('num_flex(1.234       ,4)',num_flex(1.234       ,4), '1.234'),
        ('num_flex(1.2340000   ,4)',num_flex(1.2340000   ,4), '1.234'),
        ('num_flex(0001.234    ,4)',num_flex(0001.234    ,4), '1.234'),
        ('num_flex(123456      ,5)',num_flex(123456      ,5), '123456'),
        ('num_flex(0           ,5)',num_flex(0           ,5), '0'),
        ('num_flex(00000.00000 ,5)',num_flex(00000.00000 ,5), '0'),
        ('num_flex(00000.00001 ,5)',num_flex(00000.00001 ,5), '0.00001'),
        ('num_flex(12345678901 ,5)',num_flex(12345678901 ,5), '12345678901'),
        ('num_flex(123456789.1 ,5)',num_flex(123456789.1 ,5), '123456789'),
        ('num_flex(1.234      ,-4)',num_flex(1.234       ,4), '1.234')
    ) as t ("operation","result","should_be");
    --        operation         |   result    |  should_be  | is_correct
    ----------------------------+-------------+-------------+------------
    -- num_flex(0.1234      ,4) |      0.1234 | 0.1234      | t
    -- num_flex(1.234       ,4) |       1.234 | 1.234       | t
    -- num_flex(1.2340000   ,4) |       1.234 | 1.234       | t
    -- num_flex(0001.234    ,4) |       1.234 | 1.234       | t
    -- num_flex(123456      ,5) |      123456 | 123456      | t
    -- num_flex(0           ,5) |           0 | 0           | t
    -- num_flex(00000.00000 ,5) |           0 | 0           | t
    -- num_flex(00000.00001 ,5) |     0.00001 | 0.00001     | t
    -- num_flex(12345678901 ,5) | 12345678901 | 12345678901 | t
    -- num_flex(123456789.1 ,5) |   123456789 | 123456789   | t
    -- num_flex(1.234      ,-4) |       1.234 | 1.234       | t
    --(11 rows)
    

    You can declare the precision (total number of digits) of your numeric data type in the column definition. Only digits after decimal point will be rounded. If there are too many digits before the decimal point, you'll get an error.

    The downside is that numeric(n) is actually numeric(n,0), which is dictated by the SQL standard. So if by limiting the column's number of digits to 5 you want to have 12345.0 as well as 0.12345, there's no way you can configure numeric to hold both. numeric(5) will round 0.12345 to 0, numeric(5,5) will dedicate all digits to the right of decimal point and reject 12345.

    create table test (numeric_column numeric(5));
    insert into test values (12345.123);
    table test;
    -- numeric_column
    ------------------
    --          12345
    --(1 row)
    
    insert into test values (123456.123);
    --ERROR:  numeric field overflow
    --DETAIL:  A field with precision 5, scale 0 must round to an absolute value less than 10^5.