Parse JSON with missing fields using cjson Lua module in Openresty

I am trying to parse a json payload sent via a POST request to a NGINX/Openresty location. To do so, I combined Openresty's content_by_lua_block with its cjson module like this:

# other locations above

location /test {                                                                                                               
    content_by_lua_block {                                                                                                 
        ngx.req.read_body()                                                                                            
        local data_string = ngx.req.get_body_data()                                                                    
                                                                                                                        
        local cjson = require "cjson.safe"                                                                             
        local json = cjson.decode(data_string)                                                                         
                                                                                                                        
        local endpoint_name = json['endpoint']['name']                                                             
        local payload = json['payload']                                                                            
        local source_address = json['source_address']                                                              
        local submit_date = json['submit_date']                                                                    
                                                                                                                            
        ngx.say('Parsed')                                                                                     
                                                                                                                            
    }                                                                                                                      
}

Parsing sample data containing all required fields works as expected. A correct JSON object could look like this:

{
    "payload": "the payload here",
    "submit_date": "2018-08-17 16:31:51",
    },
    "endpoint": {
        "name": "name of the endpoint here"
    },
    "source_address": "source address here",
}

However, a user might POST a differently formatted JSON object to the location. Assume a simple JSON document like

{
    "username": "JohnDoe",
    "password": "password123"
}

not containing the desired fields/keys.

According to the cjson module docs, using cjson (without its safe mode) will raise an error if invalid data is encountered. To prevent any errors being raised, I decided to use its safe mode by importing cjson.safe. This should return nil for invalid data and provide the error message instead of raising the error:

The cjson module will throw an error during JSON conversion if any invalid data is encountered. [...]

The cjson.safe module behaves identically to the cjson module, except when errors are encountered during JSON conversion. On error, the cjson_safe.encode and cjson_safe.decode functions will return nil followed by the error message.

However, I do not encounter any different error handling behavior in my case and the following traceback is shown in Openresty's error.log file:

2021/04/30 20:33:16 [error] 6176#6176: *176 lua entry thread aborted: runtime error: content_by_lua(samplesite:50):16: attempt to index field 'endpoint' (a nil value)

Which in turn results in an Internal Server Error:

<html>                                                                                                                                 
<head><title>500 Internal Server Error</title></head>                                                                                  
<body>                                                                                                                                 
<center><h1>500 Internal Server Error</h1></center>                                                                                    
<hr><center>openresty</center>                                                                                                         
</body>                                                                                                                                
</html>

I think a workaround might be writing a dedicated function for parsing the JSON data and calling it with pcall() to catch any errors. However, this would make the safe mode kind of useless. What am I missing here?

Solution

Your “simple JSON document” is a valid JSON document. The error you are facing is not related to cjson, it's a standard Lua error:

resty -e 'local t = {foo = 1}; print(t["foo"]); print(t["foo"]["bar"])'
1
ERROR: (command line -e):1: attempt to index field 'foo' (a number value)
stack traceback:
    ...

“Safeness” of cjson.safe is about parsing of malformed documents:

cjson module raises an error:

  resty -e 'print(require("cjson").decode("[1, 2, 3"))' 
  ERROR: (command line -e):1: Expected comma or array end but found T_END at character 9
  stack traceback:
      ...

cjson.safe returns nil and an error message:

resty -e 'print(require("cjson.safe").decode("[1, 2, 3"))'
nilExpected comma or array end but found T_END at character 9