Blockchain (Hyperledger sawtooth) data management

Question

When you think about designing a blockchain network using Hyperledger sawtooth, It’s important to consider how your data will grow and amplify as your business. Being familiar with on-chain and off-chain data there are certain questions that popped up in terms of data management.

1. What if the validators run out of storage?
2. What are the ways for scaling the validator who has to have a copy of all the transactions?
3. How much data one transaction generates?
4. What will be the suitable design If there is a possibility of getting a billion transactions over a period of time?
5. If that generates considerable amount of data then what are the ways for cutting down the data which is not in use anymore?
6. Can I perform CRUD operation in TSDB which sawtooth uses by default?
7. If using an IOT devices with the project then it generates a huge amount of data. How to manage this data in a decentralised and distributed way?
8. If using third party for data management then what are the efficient and cost effective methods with features like sharding, archiving, data consistency, etc..

Please clear my doubts and also correct me wherever I’m wrong with the above. Thank you.

Answer 1

When provisioning blockchain nodes, you must allocate enough storage on all the peer nodes to hold duplicate copies of the blockchain. To calculate the storage required, take the size of your transaction family payload (which is application specific) add the size of transaction and block header overhead and multiply this sum by the number of transactions you expect over the lifetime of the blockchain (or at least until the next upgrade cycle, presumably years from now).