Deserialization of simple Lua table stored as string

I'm transfering a lua table literal in a string from a web application in to PICO-8 that I'm trying to deserialize back in to a lua table in PICO-8.

The string is in the form '{"top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'

To try and keep things simple I'm only going to include lowercase characters in the strings and only strings are allowed in the nested tables.

I feel like I've got a grasp on parsing the characters, but I don't know how to keep track of where I am in the recreated data, both the depth of the structure and the index.

How is this usually done?

The catch is that as PICO-8 lua doesn't contain load or loadstring the parsing has to be done manually. The following code is using table.insert and string.sub instead of the PICO-8 equivalents because I'm using a lua REPL to help prototype this code.

Here is what I have so far with print statements what I think I need to do where.

Any help would be greatly appreciated.

test_obj = {"top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}
data_string = '{"top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'
data = nil
string = ''
level = 0
while #data_string > 0 do
 local d=string.sub(data_string,1,1)
  if stringChar(d) then
    string = string..d
  end
  if comma(d) then
    print(string)
    table.insert(data, string)
    string = ''
  end
  if openBracket(d) then
      if data == nil then
      data = {}
      print('new table')
    else
      print('insert table')
    end
    level = level + 1
    print('on level', level)
  end
  if closeBracket(d) then
    print('end of table')
    level = level - 1
    print('up to level', level)
  end
  data_string=string.sub(data_string,2)
end

Solution

Use Lua patterns to avoid parsing each character

local function read_exp_list(s)
   local exps, res = {}, {}
   local function save(v)
      exps[#exps + 1] = v
      return ('\0'):rep(#exps)
   end
   s = s:gsub('%b{}', function(s) return save{read_exp_list(s:sub(2, -2))} end) -- arrays
   s = s:gsub('"(.-)"', save)                                                   -- strings
   s = s:gsub('%-?%d+', function(s) return save(tonumber(s)) end)               -- integer numbers
   for k in s:gmatch'%z+' do
      res[#res + 1] = exps[#k]
   end
   return (table.unpack or unpack)(res)
end

local data_string = '{-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'
local obj = read_exp_list(data_string)
-- obj == {-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}

Strings must be enclosed in " and must not contain characters {}\ inside. String may be empty.
Numbers must be integer in decimal notation with optional minus.
Arrays must contain only strings, numbers and subarrays. Array may be empty.

UPDATE:
The following code only uses functions string.sub, table.insert, tonumber, type

local function is_digit(c)
   return c >= '0' and c <= '9'
end

local function read_from_string(input)
   if type(input) == 'string' then
      local data = input
      local pos = 0
      function input(undo)
         if undo then
            pos = pos - 1
         else
            pos = pos + 1
            return string.sub(data, pos, pos)
         end
      end
   end
   local c
   repeat
      c = input()
   until c ~= ' ' and c ~= ','
   if c == '"' then
      local s = ''
      repeat
         c = input()
         if c == '"' then
            return s
         end
         s = s..c
      until c == ''
   elseif c == '-' or is_digit(c) then
      local s = c
      repeat
         c = input()
         local d = is_digit(c)
         if d then
            s = s..c
         end
      until not d
      input(true)
      return tonumber(s)
   elseif c == '{' then
      local arr = {}
      local elem
      repeat
         elem = read_from_string(input)
         table.insert(arr, elem)
      until not elem
      return arr
   end
end

local data_string = '{-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'
local obj = read_from_string(data_string)
-- obj == {-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}

Strings must be enclosed in " and must not contain character \ inside. String may be empty.
Numbers must be integer in decimal notation with optional minus.
Arrays must contain only strings, numbers and subarrays. Array may be empty.