Search code examples
databasescalingshardingdistributed-databaseterminusdb

Scaling TerminusDB to multiple servers

Can TerminusDB run in a multi-server environment?

  • Either with a Sharded and Clustered database mode
  • Or can it run on Distributed database scheme?
  • Or it can only run as a single process and does not horizontally scale?

If it can scale horizontally, how?

Solution

  • You can run multiple TerminusDB instances that use the same store, by using a distributed file system that supports at least optional locking. This is the case for NFS4 and many other distributed POSIX file systems. It may also work on windows shares (SMB), but this is currently untested. Besides pointing at the correct storage directory on each server (using the TERMINUSDB_SERVER_DB_PATH environment variable or by starting the server in the appropriate directory to allow auto-discovery), no extra setup has to be done to make this work.

    TerminusDB does not support sharding. To use a database, each server instance needs to load in that full database into memory, so there's nothing gained by horizontally scaling if your goal is to reduce the memory footprint of individual instances.

    Using multiple TerminusDB instances will help you though if you want to increase query throughput, as you can just round-robin requests to a server pool. This will be especially beneficial for read-heavy workloads.