Separate string into table of words and spaces between

Question

I am currently working on a lua script that takes in a string and separates it into a table of words and the spaces + characters between the words.

Example:

-- convert this
local input = "This string, is a text!"

-- to this
local output = {
    "This", " ", "string", ", ", "is", " ", "a", " ", "text", "!"
}

I tried solving this with lua's pattern implementation, but wasn't successful so far.

Any help is highly appreciated!

Answer 1

local function splitter(input)
  local result = {}
  for non_word, word, final_non_word in input:gmatch "([^%w]*)(%w+)([^%w]*)" do
    if non_word ~= '' then
      table.insert(result, non_word)
    end
    table.insert(result, word)
    if final_non_word ~= '' then
      table.insert(result, final_non_word)
    end
  end
  return result
end