Search code examples
nearprotocol

Near indexer sync , why is it taking so much space with archive set to false?


I have a really beginner question, I'm syncing the indexer on testnet and it downloads like 30+Go (it takes ~2days)

so it seems it's downloading the whole ledger

what I don't understand is why

knowing that the "archive" option is set to false in the config.json(edited)

I thought it would only download 2.5epoch or something like that(edited)

so I guess I'm missing something fundamental here

anyone could explain this to me ? thanks a lot !


Solution

  • It is described in README section of Indexer for Explorer here https://github.com/near/near-indexer-for-explorer#syncing

    Shortly:

    • every node (including indexer) has to store the entire blockchain state, all block headers and blocks for 5 last epochs (each epoch is 42300 blocks)
    • you'd better you public backup to boost your syncing process and not wait for a couple of days
    • you need to have ~200 GB of SSD (current mainnet is ~160 GB)

    Unfortunately, there is nothing you can do to avoid syncing and heavy storage usage.

    At NEAR we're thinking about how to create a serverless indexer as a service for users. Though no ETA at the moment.