This script displays a value in a program:
string.format("%d", math.floor(self:value())) -- Where self:value() is a number like 4.1256913947
This script takes that same value and prints it to my console.
math.floor(value) -- Where value is the same number as self:value() was...it's just being sent to another function as well
The problem is the display value and the console printed value don't match up. This happens rarely and infrequently, but when a user enters data that goes over the integer slightly (ie 4.0029893417) or whatever then it will floor it to 3 in this case for no reason. I have tried math.ceil but the reverse effect happens when it's to close (ie 4.89898989) it prints 5 but displays 4. Is there a way to make it so the number perfectly truncates? Thus displaying correctly and printing correctly?
Note: The Value is unchangeable, as sometimes those extra decimals need to be there and are dealt with accordingly in other places/functions.
Thanks :)
Out of curiosity, I've performed some tests, using some of the conditions you described. Unfortunately (or fortunately) I wasn't able to reproduce the problem you're talking about. The snippet below adds and subtracts random numbers, so that the result is around 4 and then compares both methods of printing (compares by produced string):
for i=1,10000000 do
local value = 3.9 + math.random() * 0.2
local s1 = string.format("%d", math.floor(value))
local s2 = tostring(math.floor(value))
if s1 ~= s2 then print "Error" end
end
As you can guess, 'Error' doesn't show up.
I've made yet another experiment:
local l = 4
for i=1,100000 do
local k = math.pow(10,-i)
local rand = math.random() * 0.1
local subtracted = l - k - rand
if math.floor(subtracted + k + rand) < 4 then
print "error"
end
end
By adding very small number I was hoping to get similar result to what you describe, but again, problem doesn't show up.
So I must conclude that the problem probably lays in the difference between function self:value() and variable value (they indeed have different values sometimes).