Search code examples
calculated-columnsdecimalformatrounding-error

Calculations being being rounded SQL Server 2012


I am trying to calculate some performance metrics as [RATE] in SQL but no matter what I do I am only getting integer values in return.

In my dataset I have a Numerator and a Denominator both stored as integers that are broken down into groups and subsets. The rate is calculated slightly differently depending on the group and subset. For group 1 the calculation is simply N/D. For group 2 the calculation is (N*1000)/D with the exception of 1 subset which is calculated (N*10000)/D.

I wrote the query as:

SELECT [Group]
      ,[SubSet]
      ,[Numerator] N
      ,[Denominator] D
      ,CASE WHEN D=0 Then NULL
        WHEN [Group]='G1' THEN [N]/[D]
        WHEN [SubSet]='S2' THEN ([N]*10000)/[D]
        WHEN [SubSet] NOT LIKE 'S2%' AND [G] NOT LIKE 'G1%' THEN ([N]*1000)/[D] as [RATE]

No matter what I do the outcome variables are integers. I tried formatting RATE as varchar, decimal, and float with no success. I tried changing N and D's format to varchar, decimal, and float as well. I tried changing the equations from (N*1000)/D to (N/D)*1000 but still no effect.

What am I missing/doing wrong?


Solution

  • The problem you are having is because SQL is doing integer division, which will only return whole numbers. To get a decimal return value you must have at least one value as a decimal.

    Try this:

    (CAST([N] as decimal(12,6))/[D]) * 1000
    

    Adjust decimal(12,6) based on the precision you are expecting. 12,6 will return a decimal with 6 digits after the decimal point. If you wanted only 2 decimal places use 16,2.

    If you then want to round the calculated value you will need to make use of the ROUND function in SQL.

    Round to the second decimal place:

    ROUND((CAST([N] as decimal(12,6))/[D]) * 1000, 2)