Is it possible to stream serialized data into file with flatbuffers?

Question

I am making an online game in node.js and trying to save a game replay on my game's server. I am using flatbuffers to serialize data for client-server communication and I thought it would be cool to save my game's state frame by frame in the file.

I created the following table in my .fbr file

table Entity {
    id: ushort;
    pos: Vec2;
}

table Frame {
    entities: [Entity];
}

table Replay {
    frames: [Frame];
}

Is there a way to write all game state frames to the file on the fly? I know that I could just buffer N frames and save them in separate replay files, but i feel that there should be a better way. I want my replay to be in a single file, otherwise it would be very inconvenient to use it afterwards.

Answer 1

The best way to do this is to make sure each individual FlatBuffer is a "size prefixed buffer" (it has a 32-bit size ahead of the actual buffer), which in JS you can create by calling this instead of the usual finish function: https://github.com/google/flatbuffers/blob/6da1cf79d90eb242e7da5318241d42279a3df3ba/js/flatbuffers.js#L715

Then you write these buffers one after the other into an open file. When reading, because the buffers start with a size, you can process them one by one. I don't see any helper functions for that in JS, so you may have to do this yourself: read 4 bytes to find out the size, then read size bytes, repeat.