Search code examples
azureazure-service-fabricservice-fabric-stateful

How is data in Reliable Dictionary in Azure Service Fabric persisted to disk


How does Azure Reliable Dictionary actually handles data persistence. Is it like SQL which persists mostly all data excepts indexes(i.e. key in our case) on disk or is like hashmap which have everything in memory(i.e. both key and value) and just keeps a backup on disk


Solution

  • The current implementation of Reliable Collections store the entire set of data in the memory, and a copy serialized in the disk.

    It is required to stay in memory because the data stored in the disks are serialized and a lookup on this data would take too long to deserialize to retrieve the data.

    From one of the Q&A sessions they mentioned that are in the plans to have an alternative solution where hot data would be in memory and cold in the disk, but it was just an item in the backlog at time, not sure if they will implement anytime soon.

    The following link will also give some extra insights about the collection storage:

    Capacity Limit of Reliable Collections in Stateful services

    Azure Service Fabric reliable collections and memory

    How data serialization affects an application upgrade

    Downsides of CommitAsync() w/o any changes to collection