etcd - save data to custom database

Trying to figure out how to create a simple distributed database

etcd is use to consensus metadata in Kubernetes. I can see Dgraph BadgerDB and other key value stores are using etcd, but I don't know quite how they are using it. Update: It looks like they are using a raft subset of etcd.

My question:

etcd is for storing metadata and not data as such - is it possible/recommended to combine etcd with another key value store to handle large data?

I have also looked at hashicorp raft

Solution

The etcd service was conceived to handle metadata: retrieve one key, get some data wich usually in the kilobytes range. Not megabytes.

You cannot "offlad" etcd to another database / datastore. Etcd need to have its data with low latency on the majority of nodes.

The biggest criticism about etcd are hardware requirements: 8GB RAM x 3 machines might be too much for some use cases.

Is it etcd good for you? It depends a lot on:

Your workload (mostly read-oriented than write-oriented)
How many requests you need from etcd in a short time period: do not 'fire' too many requests at once to etcd or you might trigger a leader election.
And your data size: use --auto-compaction enabled to keep disk usage as low as possible removing old data versions.