Search code examples
javascriptlua

Lua in JS runtime (Fengari): String representation of large integers has trailing ".0"

So I have a slightly complex problem. To help with the overall issue, here's the background:

I'm converting an app to javascript, with part of the app using lua to interpret templates. I don't control these templates. However, the template parser is in lua, and it looks at the template string for lines beginning with % or lines that contain <% … %> tags. For example:

this is output as is
% for index, value in ipairs{1,2,3} do -- this is lua code that doesn't output
index:<% index %>; value:<% value %>
% end

would output:

this is output as is
index:1; value:1
index:2; value:2
index:3; value:3

lines that don't start with % get wrapped in a lua helper append(line_contents) function, and if they have tags, such as foo<% tag_variable %>bar it gets transformed like append('foo'..(tag_variable)..'bar'). Lines that start with % get transformed to lua code (essentially eval'd). This code works well, and I can control the logic that parses the template, and modify the append helper. (But not the template code)

So the main issue happens when the template code is working with ~10-digit numbers (integers). In C-lua the string representation of those is just the integer part when the value is used in a template tag. However, in the Fengari runtime of Lua in JS, the string representation of some of those numbers sometimes has a trailing .0. This means output varies between the runtimes. I'm trying to figure out ways to fix this.

I believe that this is caused because Fengari relies on the JS Number.prototype.toString function to format numbers as strings when needed. And it looks like this is an oddity of that function, that some point some 10+ digit numbers start having trailing decimal 0s. This is defined behavior for lua, not javascript, and happens when the internal representation uses a float type, as Egor mentions in the comments. I've googled around and can't find any mention of this problem exactly, but believe it's related to the float representation of numbers. I could use Number.prototype.toFixed, but that doesn't fix the problem globally. I also want to understand better why this is happening, as well as try to find a solution. I am considering somehow overwriting Number.prototype.toString if that would work universally, but I know that would be tricky, if it is even possible... Additionally, I could change how the template tags are wrapped and add a formatting helper around the output, and that might fix the issue in some cases, but it doesn't help cases where numbers are getting concatenated by template code... So how might I approach this?

Reference: here's the spec on how numbers are represented as strings in JS: https://www.ecma-international.org/ecma-262/7.0/#sec-tostring-applied-to-the-number-type. But I'm apparently too tired to understand that dense math definition today :).

Solution

  • Answered on the Fengari GitHub page here: https://github.com/fengari-lua/fengari/issues/183

    fengari's behaviour intentionally matches Lua 5.3 here. See LUA_COMPAT_FLOATSTRING and https://www.lua.org/source/5.3/lobject.c.html#luaO_tostring and the matching fengari code:

    fengari/src/lobject.js (Lines 592 to 594 in 0c9631c)

     if (!LUA_COMPAT_FLOATSTRING && /^[-0123456789]+$/.test(str)) {  /* looks like an int? */ 
     str += '.0'; /* adds '.0' to result: lua_getlocaledecpoint removed as optimisation */ 
 }

    This is documented here:

    fengari/src/luaconf.js (Lines 123 to 129 in 0c9631c)

     /* 
 @@ LUA_COMPAT_FLOATSTRING makes Lua format integral floats without a 
 @@ a float mark ('.0'). 
 ** This macro is not on by default even in compatibility mode, 
 ** because this is not really an incompatibility. 
 */ 
 const LUA_COMPAT_FLOATSTRING = conf.LUA_COMPAT_FLOATSTRING || false;

    This can be changed by setting process.env.FENGARICONF with a JSON string defining LUA_COMPAT_FLOATSTRING