Confusion with debug.getlocal in Lua

I saw here how to insert local variables in a table using the debug.getlocal function in Lua (5.1).

function locals()
   local variables = {}
   local idx = 1
   while true do
     local ln, lv = debug.getlocal(2, idx)
     if ln ~= nil then
       variables[ln] = lv
     else
       break
     end
     idx = 1 + idx
   end
   return variables
 end

However, when I try to return the created table and access it's entries, it doesn't work.

function test1()
    local v = 'I am a local!'
    return locals()
end

print(test1().v) -- nil

After some trail and error, I noticed that binding the table to a variable before returning, or simply adding parentheses fixes the behavior:

function test2()
    local v = 'I am a local!'
    return (locals())
end

print(test2().v) -- 'I am a local!'

This is very confusing me. Why are these two programs in any way different? What am I not understanding? Does the fact that locals() is in a tail call position make any difference?

Solution

I guess what confuses you is the proper tail call feature of lua.

To understand this, we modify your locals function, making it accept one argument as the level stack used in call to debug.getlocal. (I'm using Lua 5.3.3)

-- Get local variables with stack level 'level'.
function locals(level)
    local variables = {}
    local idx = 1
    while true do
        local ln, lv = debug.getlocal(level, idx)
        if ln ~= nil then
            variables[ln] = lv
        else
            break
        end
        idx = 1 + idx
    end
    return variables
end

Then we modify your test functions, adding the same argument, and add a test3 function for reference.

function test1(level)
    local v = 'I am a local!'
    return locals(level)
end

function test2(level)
    local v = 'I am a local!'
    return (locals(level))
end

function test3(level)
    local v = 'I am a local!'
    local a = locals(level)
    return a
end

Finally we add some code to run the tests.

local function printTable(t)
    -- print(t)
    for k, v in pairs(t) do
        print(string.format("key = %s, val = %s.", k, v))
    end
end

for level = 1, 3 do
    print("==== Stack level: " .. tostring(level))
    for num = 1, 3 do
        print(string.format("What test%d returns: ", num))
        printTable(_G[(string.format("test%d", num))](level))
        print("")
    end
end

The code above runs the test functions with different stack level and print the key-value pairs returned. My result is as follows:

==== Stack level: 1
What test1 returns: 
key = variables, val = table: 0x7fa14bc081e0.
key = idx, val = 3.
key = level, val = 1.

What test2 returns: 
key = variables, val = table: 0x7fa14bc08220.
key = idx, val = 3.
key = level, val = 1.

What test3 returns: 
key = variables, val = table: 0x7fa14bc088b0.
key = idx, val = 3.
key = level, val = 1.

==== Stack level: 2
What test1 returns: 
key = (for step), val = 1.
key = (for limit), val = 3.
key = (for index), val = 1.
key = level, val = 2.
key = printTable, val = function: 0x7fa14bc08360.
key = (*temporary), val = function: 0x7fa14bc08360.
key = num, val = 1.

What test2 returns: 
key = level, val = 2.
key = v, val = I am a local!.

What test3 returns: 
key = level, val = 2.
key = v, val = I am a local!.

==== Stack level: 3
What test1 returns: 
key = (*temporary), val = function: 0x109f5a070.

What test2 returns: 
key = (for step), val = 1.
key = (for limit), val = 3.
key = (for index), val = 2.
key = level, val = 3.
key = printTable, val = function: 0x7fa14bc08360.
key = (*temporary), val = function: 0x7fa14bc08360.
key = num, val = 2.

What test3 returns: 
key = (for step), val = 1.
key = (for limit), val = 3.
key = (for index), val = 3.
key = level, val = 3.
key = printTable, val = function: 0x7fa14bc08360.
key = (*temporary), val = function: 0x7fa14bc08360.
key = num, val = 3.

When level is 1, locals works well to give its own local variables. But when level is 2, test1 returns variables of the outer scope, whereas test2 and test3 give the result you expect. For stack level 3 test2 and test3 return something like test1 at stack level 2. So it seems test1 skips a stack level, and the only explanation I could think of is the proper tail call.

According to PIL (the link I provide at the beginning), a proper tail call will never cause the stack to overflow, which I take as doing the call in some inline way. If I'm right with that, this explains the skipping behaviour of test1's return statement, because that's a proper tail call, and the only one in the 3 test functions.