Search code examples
node.jshyperledger-sawtooth

Recommended way to serialize state data in sawtooth node.js transaction processor?


According to the documentation: "It is critical to select a serialization scheme which is deterministic across executions of the transaction, across platforms, and across versions of the serialization framework. Data structures which don’t enforce ordered serialization (e.g. sets, maps, dicts) should be avoided." https://sawtooth.hyperledger.org/docs/core/releases/1.2.6/architecture/global_state.html?highlight=deterministic

Because in javascript most variables are maps, it's not clear how one should should store data.

Thus, storing general JSON data would seem to be a bad idea, as in:

context.setState({[address]:Buffer.from(JSON.stringify({key1:"value", key2:"value"}))});

Although JSON.stringify is to a certain extent deterministic (Is JSON.stringify() deterministic in V8?)

Also cbor and protobuff might not be recommended because they do not enforce an ordering:

"The CBOR data model for maps does not allow ascribing semantics to the order of the key/value pairs in the map representation." https://www.rfc-editor.org/rfc/rfc7049

"By default, repeated invocations of serialization methods on the same protocol buffer message instance may not return the same byte output; i.e. the default serialization is not deterministic." https://developers.google.com/protocol-buffers/docs/encoding

"Wire format ordering and map iteration ordering of map values is undefined, so you cannot rely on your map items being in a particular order." https://developers.google.com/protocol-buffers/docs/proto3#maps

From the documentation the only examples I've seen are storing a String with a specific format: https://github.com/hyperledger/sawtooth-sdk-javascript/blob/master/examples/xo/state.js

Or storing a single cbor encoded value: https://github.com/hyperledger/sawtooth-sdk-javascript/blob/master/examples/intkey/handler.js

However, both approaches seem very limited.


Solution

  • You should either develop a helper function that do a deterministic version of stringify or you should consider using a library that provides a deterministic stringify like this one: json-stringify-deterministic (https://github.com/kikobeats/json-stringify-deterministic)