Search code examples
schemaflatbuffers

Flatbuffers: How to allow multiple types for a single field


I'm writing a communication protocol schema for a list of parameters which can be of multiple values: uint64, float64, string or bool.

How can I set a table field to a union of multiple primitive scalar & non-scalar primitive type?

I've already tried using a union of those types, but I end up with the following error when building:

$ schemas/foobar.fbs:28: 0: error: type referenced but not defined
  (check namespace): uint64, originally at: schemas/request.fbs:5

Here's the schema in its current state:

namespace Foobar;

enum RequestCode : uint16 { Noop, Get, Set, BulkGet, BulkSet }

union ParameterValue { uint64, float64, bool, string }

table Parameter {
  name:string;
  value:ParameterValue;
  unit:string;
}

table Request {
  code:RequestCode = Noop;
  payload:[Parameter];
}

table Result {
  request:Request;
  success:bool = true;
  payload:[Parameter];
}

The end result I'm looking for is the Request and Result tables to contain a list of parameters, where a parameter contains a name and value, and optionally the units.

Thx in advance!

Post-answer solution: Here's what I came up with in the end, thx to Aardappel.

namespace foobar;

enum RequestCode : uint16 { Noop, Get, Set, BulkGet, BulkSet }

union ValueType { UnsignedInteger, SignedInteger, RealNumber, Boolean, Text }

table UnsignedInteger {
  value:uint64 = 0;
}

table SignedInteger {
  value:int64 = 0;
}

table RealNumber {
  value:float64 = 0.0;
}

table Boolean {
  value:bool = false;
}

table Text {
  value:string (required);
}

table Parameter {
  name:string (required);
  valueType:ValueType;
  unit:string;
}

table Request {
  code:RequestCode = Noop;
  payload:[Parameter];
}

table Result {
  request:Request (required);
  success:bool = true;
  payload:[Parameter];
}

Solution

  • You currently can't put scalars directly in a union, so you'd have to wrap these in a table or a struct, where struct would likely be the most efficient, e.g.

    struct UInt64 { u:uint64 }
    union ParameterValue { UInt64, Float64, Bool, string }
    

    This is because a union must be uniformly the same size, so it only allows types to which you can have an offset.

    Generally though, FlatBuffers is a strongly typed system, and the schema you are creating here is undoing that by emulating dynamically typed data, since your data is essentially a list of (string, any type) pairs. You may be better off with a system designed for this particular use case, such as FlexBuffers (https://google.github.io/flatbuffers/flexbuffers.html, currently only C++) which explicitly has a map type that is all string -> any type pairs.

    Of course, even better is to not store data so generically, but instead make a new schema for each type of request and response you have, and make parameter names into fields, rather than serialized data. This is by far the most efficient, and type safe.