How to set a flatbuffers table field with bytes array

Question

I want set a flatbuffers table field with bytes value.

What I have managed so far, without success, it is the following.

flatbuffers schema:

namespace sint.bl;

table Response {
    id:short;
    body:[byte];
}

python sample code:

import flatbuffers
import pickle
import sint.bl.Request
import sint.bl.Response

my_dict = {
        'a': 1
}

my_bytes = pickle.dumps(my_dict)

builder = flatbuffers.Builder(1024)

sint.bl.Response.ResponseStart(builder)
sint.bl.Response.ResponseAddId(builder, 100)

# this line throws the exception:
# ValueError: invalid literal for int() 
# with base 10: b'\x80\x03}q\x00X\x01\x00\x00\x00aq\x01K\x01s.'
sint.bl.Response.ResponseAddBody(builder, my_bytes)

response = sint.bl.Response.ResponseEnd(builder)

builder.Finish(response)

response_pdu = builder.Output()

What is it the correct way to manage bytes encoded fields with flatbuffers?

Answer 1

The argument to ResponseAddBody is an offset to the serialized byte vector (the int in your error), not the bytes object directly. This needs to be serialized before the table.

So, right after you create builder, call builder.CreateByteVector(my_bytes), the result of which you pass to ResponseAddBody later.

Alternatively, here's how to create any vector manually (select Python, search for inventory): https://google.github.io/flatbuffers/flatbuffers_guide_tutorial.html

Note, you appear to be using 2 serialization systems, pickle and FlatBuffers. You'd be better off encoding the picked data directly in FlatBuffers, using e.g. table Foo { a:int } instead of your dict, or if it has to be an open-ended dict, a vector of table KeyValue { key:string; value:int; } or similar. Or a union, depending on your use case.