Search code examples
tarantooltarantool-cartridge

How to return the table with tuples of the SELECT()?


It tarantool-cartridge. Caller function:

local function http_redirections(req)
    log.info("-- redirections started")

    local b_id = tostring(req)
    local bucket_id = vshard.router.bucket_id_mpcrc32(b_id)
    log.info("-- bucket_id: %s", bucket_id)
    local answer, error = err_vshard_router:pcall(
        vshard.router.call,
        bucket_id,
        'read',
        'redirections'
    )

    for k, v in pairs(answer) do log.info(" http: key: %s value: %s", k, v) end
end

Handler:

local function redirections(req)
    log.info("-- redirections handler started")
    local response = {}
    local answer = {}
    response = box.space.space_name:select()
    for k, v in pairs(response) do answer[k] = v end
    for k, v in pairs(answer) do log.info("key: %s value: %s", k, v) end
    log.info("-- response size: %d", #answer)
    return answer
end

Result in the calling function: http: key: 1 value: table: 0x418e3d10 http: key: 2 value Result in handler OK: key = value from select(). Why does a table from a table return to the calling function ?


Solution

  • In short, you don't have any legal ways to return tuples from netbox/vshard call.

    How does it work:

    -- You have a tuple
    tuple = box.tuple.new({1, 2, 3})
    tarantool> box.tuple.is(tuple) -- It's tuple, cdata object
    ---
    - true
    ...
    -- In order to transfer your tuple via network you need convert
    -- them to binary form
    msgpack = require('msgpack')
    tarantool> msgpack.encode(tuple)
    ---
    - !!binary kwECAw==
    ...
    -- But you don't know after that it was a tuple or simple lua table
    tarantool> msgpack.encode({1, 2, 3}) -- The same, but input is a lua table
    ---
    - !!binary kwECAw==
    ...
    -- After receive you need to decode you tuple/table back to lua object
    tarantool> msgpack.decode(msgpack.encode({1, 2, 3}))
    ---
    - [1, 2, 3]
    - 5
    ...
    
    tarantool> msgpack.decode(msgpack.encode(tuple))
    ---
    - [1, 2, 3]
    - 5
    ...
    tarantool> box.tuple.is(msgpack.decode(msgpack.encode(tuple)))
    ---
    - false
    ...
    

    But if you use connection you could directly call "select/insert/..." functions directly. Since Tarantool 2.2 such tuples even have a format. For details see issue.

    box.cfg{listen = 3302}
    box.schema.user.grant('guest','read, write, execute', 'space')
    box.schema.user.grant('guest', 'create', 'space')
    
    box.schema.create_space("named", {format = {{name = "id"}}})
    box.space.named:create_index('id', {parts = {{1, 'unsigned'}}})
    box.space.named:insert({1})
    require('net.box').connect('localhost', 3302).space.named:get(1).id
    
    Result:
    
    tarantool> require('net.box').connect('localhost', 3302).space.named:get(1).id
    ---
    - 1
    ...
    

    If you need to convert lua tables to tuples you could use box.tuple.new() function. But it will be tuples without format.

    tarantool> tbl = {1, 2, 3}
    ---
    ...
    
    tarantool> box.tuple.is(tbl)
    ---
    - false
    ...
    
    tarantool> tuple = box.tuple.new(tbl)
    ---
    ...
    
    tarantool> box.tuple.is(tuple)
    ---
    - true
    ...
    

    And here I should finish my answer. But you have illegal way to return formatted tuples from netbox call.

    netbox = require('net.box')
    netboxlib = require('net.box.lib')
    buffer = require('buffer')
    msgpackffi = require('msgpackffi')
    msgpack = require('msgpack')
    ffi = require('ffi')
    
    box.cfg{listen = 3301}
    box.schema.user.passwd('admin', 'test')
    
    s = box.schema.space.create('test')
    s:format({{name = 'id', type = 'string'}, {name = 'value', type = 'string'}})
    s:create_index('pk')
    s:replace({'key', 'value'})
    c = netbox.connect('admin:test@localhost:3301')
    function get()
        return s:select()
    end
    
    call_buffer = buffer.ibuf()
    c:call('get', {}, {buffer = call_buffer, skip_header = true})
    len, call_buffer.rpos = msgpack.decode_array_header(call_buffer.rpos, call_buffer:size())
    buf = buffer.ibuf()
    encode_fix = msgpackffi.internal.encode_fix
    encode_r = msgpackffi.internal.encode_r
    encode_fix(buf, 0x80, 1)
    encode_r(buf, 0x30, 1)
    wpos = buf:alloc(call_buffer:size())
    ffi.copy(wpos, call_buffer.rpos, call_buffer:size())
    
    tarantool> netboxlib.decode_select(buf.rpos, nil, c.space.test._format_cdata)
    ---
    - - ['key', 'value']
    - 'cdata<char *>: 0x010586803e'
    ...
    tarantool> tuples = netboxlib.decode_select(buf.rpos, nil, c.space.test._format_cdata)
    ---
    ...
    
    tarantool> tuples 
    ---
    - - ['key', 'value']
    ...
    
    tarantool> tuples[1]
    ---
    - ['key', 'value']
    ...
    
    tarantool> tuples[1]:tomap()
    ---
    - 1: key
      2: value
      value: value
      id: key
    ...
    

    This code works since Tarantool 2.2 and easily could be broken in future as it uses some internal Tarantool functions. I don't recommend you to use them. I hope it will be changed after this issue will be resolved.