Search code examples
luaudfaerospikeaql

AQL different results from stream UDF depending on output style (table, json)


I'm trying to create aggregation (map | reduce) with UDF but something is wrong on the very begining. In Aerospike I have a set with bin 'u' (secondary index) and bin 'v' which is a list of objects (auctions with transactions lists and other auction data) and I have a stream UDF to aggregate internal structure of 'v':

function trans_sum_by_years(s)
    local function transform(rec)

        local l = map()
        local x = map()
        local trans, auctions = 0, 0


        for i in list.iterator(rec['v'] or list()) do
            auctions = auctions + 1
            for t in list.iterator(i['t'] or list()) do
                trans = trans + 1
                date = os.date("*t", t['ts'])


                if l[date['year']] ~= nil then
                    l[date['year']] = l[date['year']] + t['price'] * t['qty']
                else
                    l[date['year']] = t['price'] * t['qty']
                end
            end
        end

        x.auctions = auctions
        x.trans = trans
        x.v = l
        return x
    end

    return s : map(transform)

end

The problem is that output is very diffrent depending on setting output on table or json. In first case it seems everything is OK:

{"trans":594, "auctions":15, "v":{2010:1131030}}            
{"trans":468, "auctions":68, "v":{2011:1472976, 2012:5188}} 
......

On second I get empty object from internal record aggregation. { "trans_sum_b...": { "trans": 389, "auctions": 89, "v": {} } }, { "trans_sum_b...": { "trans": 542, "auctions": 30, "v": {} } } .....

I prefer json output and wasted couple hours to find out why I get empty 'v' field without success. So my question is "what the hell is going on" ;-) If my code is correct, what is wrong with the json output, that I don't see the results. If my code is wrong, why it's wrong and why table output results with what I need.


Solution

  • The answer is simple as hell. But I'm new in Aerospike/Lua and I don't trust my knowledge so I searched for error everywhere but within AQL/UDF area. The problem is more fundamental and interferes with the specification of the JSON itself.

    Keys in JSON have to be strings! So tostring(date['year']) solves problem.

    Other question is does it is a bug or a feature :-) If Aerospike's map type allow integer keys should there be automatic key conversion from integer to string to satisfy JSON specification or not? IMHO there should be but probably some people disagree claiming that map type is not for integer keys...