Search code examples
graphdhtkademlia

Is a DHT (kademlia) able to reliably store chains of values?


I'm looking at implementing a DHT where the data items are chained by having a successor address added to the values stored, if each node can have one of three ordered states: Empty -> Data -> Data and Successor Address will all peers get a consistent and correct ordering? or are permanent forkings a possibility here?


Solution

  • In principle it's possible with some restrictions and support from the nodes.

    To deal with multiple versions you need value versioning. To reliably increment them without collisions you want a single originator. To ensure a single originator you will have to sign the data. Signed data is usually stored under keys derived from the public key. So the searching node will either have to acquire the public key some way or you will need another indirection to resolve human-readable keys to public keys.

    DHT.put("keyword", Pubkey)
    DHT.get("keyword") => List<Pubkey>
    
    DHT.put(Pubkey, Tuple<Value, ForwardPointer, VersionNumber>, Pubkey, Signature)
    DHT.get(Pubkey) => List<Tuple<Tuple<Value, ForwardPointer, VersionNumber>, Signature>>
    

    Note that the first argument will always be hashed. Also note that the APIs are asymmetric, returning lists and adding additional parameters for the target nodes to process and verify.

    I.e. storing nodes have to do a little more work than just being "dumb key-value storage"

    Edit: In your specific case you can probably skip the version number and use the absence/presence of the forward pointer as implicit version increment.


    In principle you can implement any data structure on top of a DHT. all you need is storage and pointers to other keys. E.g. a list can either be implemented as mutable nodes which change their forward pointers or immutable nodes + one mutable head pointer.

    For some faster traversal sorted trees or skip lists may also be worth considering.