Storing nested tables in nginx shared memory

Question

I am working with open-resty and lua to create a server for redirecting requests. Redirects are done based on some data from a lua datatree structure (nested tables)

I am looking for a way to populate this data once on startup and after that share the data between workers.

ngx.ctx can save arbitrary data but lasts only during request. Shared dict lasts till the end but can only save list of primitives.

I've read that it is possible to share data across lua modules. Because modules get instantiated only once at startup. The code is something like this

local _M = {}

local data = {
    dog = {"value1", "value4"},
    cat = {"value2", "value5"},
    pig = {"value3", "value6"}
}


function _M.get_age(name)
    return data[name]
end

return _M

and then in nginx.conf

location /lua {
    content_by_lua_block {
        local mydata = require "mydata"
        ngx.say(mydata.get_age("dog"))
    }
}

Is this third possibility thread safe? Is there something else which can achieve this?

There is not a lot of documentation on this, that is why posted it here. Any info would help, Thank you

Answer 1

You can populate your data in init_by_lua, and access it later on. In your case initialization of mydata module can be achieved by:

init_by_lua_block {
     require "mydata"
}

init_by_lua runs once during nginx startup, and then the process it run in forks into workers, so each of them contain an independent copy of this data.

Workers are single-threaded, so you can safely access your data.

---

Now, if you want to modify your configuration at runtime, without reloading nginx, then it gets a bit more complicated. Each worker is independent, but we can use ngx.shared.DICT to propagate changes. Depending on your requirements there are two solutions you can use:

1. After each change put your configuration into shared dictionary. Create a timer that periodically reloads worker's configuration from this shared cache.
2. After each change put your configuration into shared dictionary, along with current timetamp or version number. On each request in a worker check whether this timestamp/version is never than the one cached locally - if it is then deserialize this configuration and cache it locally.

If you have an API that should be usable then you can use lua-resty-lock to create cross-worker critical sections that synchronize modification.