Search code examples
.netmathexpressionprecisiondelphi-prism

Output precision is reduced to the precision of the operands

I am getting warning message on the following expression.

(400*ev.PageBounds.Width)/2400

Warning message is Output precision is reduced to the precision of the operands.

Why? Can I just ignore it?

Thanks,

Solution

  • Just use operands of wider precision:

    (400.0*ev.PageBounds.Width)/2400.0

    or cast them:

    (((double)400)*v.PageBounds.Width)/(double)2400

    etc. Note that it is the 400 what dictates the type of the result, you may skip casting the 2400

    Edit: no, in general you should not ignore it. The message warns you that dividing BLAH by 2400 may result in fractions that would be truncated. Ie. 3500 / 2400 = 1. It would be done in such way, when all of your operands are integral, for example:

    2 * 4 / 10 * 10 ==== 0

    while

    2 * 4.0 / 10 * 10 ==== 8