I am building an application in a micro service architecture . So I have my different business models running on different microservices.
Microservices are using graph and document databases.
What I have to do is, I need to keep all audit logs about the objects whenever they were changed. There are couple of ways to do this,two I thought of :
- Store audit logs in the each databases whenever something changes to object.
- Instead of having it localized, make it to a central repository where we can see all the audits for whole application as behind the scenes application is served by micro services but at front this is just one app for the users and also for us. Would elastic search be used for this purpose of long term storage ? or we have other solutions ?
Which other ways are the best practices that I must follow. My objective in the end is to the when what was changed in the object by whom.
Cheers!
General recommendation is not to use ES as your authoritative data store. If you want 99.99% reliability for the audit data store it somewhere else, and index in ES when you need its searching abilities.
In my experience ES is quite resilient, still I keep in mind its storage is not that polished comparing to well known relational DBs or Cassandra/HDFS and I would not store important data there.
Also keep in mind ES index in not very flexible, if you want to heavily rescale your cluster or to change field mapping you may have to reindex everything. Newer versions of ES offer "Reindex API", still it's weak point.