Search code examples
flatbuffers

Queries on schema and JSON data conversion


We already have flatbuffer library embedded in our software code for simple schemas with JSON output data generation.

More update: We are generating the header files using flatc compiler against the schema and integrate these files inside of our code along with FB library for further serialization/deserialization.

Now we also need to have the following schema tree to be supported.

namespace SampleNS;

/// user defined key value pairs to add custom metadata
/// key namespacing is the responsibility of the user
table KeyValue {
    key:string (key, required); 
    value:string (required);
}

enum SchemaVersion:byte {
    V1,
    V2
}

table Sometable {
    value1:ubyte;
    value2:ushort (key);
}

table ComponentData {
    inputs: [Sometable];
    outputs: [Sometable];
}

table Node {
    name:string (key);
    
    /// IO definition
    data:ComponentData;
            
    /// nested child
    child:[Components];
}

table Components {
    type:ubyte;
    index:ubyte;
    nodes:[Node];
}

table GroupMasterData {

    schemaversion:SchemaVersion = sampleNS::SchemaVersion::V1;

    metainfo:[KeyValue];

    /// List of expected components in the system
    components:[Components];
}

root_type GroupMasterData;

As from above, table Components is nested recursively. The intention is components may have childs that have the same fields.

I have few queries:

  • Flatc didnt gave me any error during schema compilation for such recursive nested tables. But is this supported during the field access for such tables?
  • I tried to generate a sample json data file based on above data but I could not see the field for schemaversion. I learned FB doesn't serialize the default values. so, I removed the default value that I assigned in the schema. But, it still doesnt write into the json data file. On this I also learned we can forcefully write into the file using force_defaults option. I don't know where is this is to be put: in the attribute or elsewhere?
  • Can I create a struct of enum field?

Is their any API to set Flatbuffer options that we otherwise pass to the compiler arguments? or if not, may be I think we have to tinker with the FB library code. Please suggest.

Method 1: In our serialization method, we do this:

flatbuffers::Parser* parser = new flatbuffers::Parser();
parser->opts.output_default_scalars_in_json = true;

Is this the right method or should I use any other API?


Solution

    • Yes, trees (and even DAG) structures are fully supported. The type definition is recursive, but the data will eventually have leaf nodes with an empty vector of children, presumably.
    • The integer value for V1 is 0, and that is also the default value for all fields with no explicit default assigned. Use --defaults-json to see this field when converting. Note that explicit versions in a schema is an anti-pattern, since schemas are naturally evolvable without breaking backwards compatibility.
    • You can put enum fields in structs, yes. Is that what you mean?