Async auditing with JaVers

Question

I need to audit changes to some entities in our application and am thinking of using JaVers. I like the support for interrogating the audit data provided by JaVers. Hibernate Envers looks good, but it stores data in the same DB.

Here are my requirements:

• async logging - for minimal performance impact
• store audit data in a different db - performance reasons as well

As far as I can see JaVers is not designed for the above, but seems possible to adapt to achieve the above. Here's how:

• JaVers actually allows data to be stored in a different DB. You can provide a connection to any DB really. It's not how it's intended, but it works. Code below (note connectionProvider which can provide a connection to any DB):

'

final Connection dbConnection =
            DriverManager.getConnection("jdbc:mysql://localhost:3306/javers", "root", "root");

ConnectionProvider connectionProvider = new ConnectionProvider() {
    @Override
    public Connection getConnection() {
        //suitable only for testing!
        return dbConnection;
    }
};
JaversSqlRepository sqlRepository = SqlRepositoryBuilder
            .sqlRepository()
            .withConnectionProvider(connectionProvider)
            .withDialect(DialectName.MYSQL).build();

• The async can be achieved by moving the execution of the JaVers commit into a thread/executor. The challenge with that is that if the execution takes too long, it could be that the object changes before it's logged. There are 2 solutions I can think of here:
  • we could create a snapshot of the object (e.g. serialize it to JSON or the like) and pass that to a Thread to log it.
  • we provide our custom implementation of Javers Repository which processes the differences in the current thread, and then passes the Snapshot objects to be persisted in another thread. This way we'd only do reading from DB in the application thread, and do writing (which is generally more costly performance wise) in the Auditing thread.

QUESTIONS:

• am I missing anything here? Could this work?
• Does JaVers have support to create a snapshot of the object which then can be moved to another thread. It does it internally somewhere, so maybe it's something we could use.

JUST FYI: Not relevant for the question, but here are some other challenges I can think of and how I'm planning to solve them:

• due to not doing audits in the same transaction, as if the transaction fails, it'd make audit rollback complex. So we need to audit only objects that were successfully committed. I intend to do that by using a Hibernate Interceptor, listening to the afterTransactionCompletion and only committing objects updated by that transaction.
• In case of lazy loaded objects, I could see how, if we're trying to access them once the transaction is finished, it might be that the lazy loaded props can't be accessed (as the session might be closed too) - don't know how to fix this, but it might not be an issue as I think we're loading eager most props.

Answer 1

Interesting question. First the démenti. All JaVers core modules are designed to decouple audit data from application data. As you mentioned, user provides a ConnectionProvider to be used by JaVers. It could be any database you want.

What are not designed to use with multiple DB are Spring integration modules for SQL, so javers-spring-jpa and javers-spring-boot-starter-sql. They just cover most common scenario so the same DB for application and JaVers.

You are right about lack of async commit. Fortunately, it can be implemented only in JaversCore without changing the Repositories.

The API could be:

CompletableFuture<Commit> javers.commitAsync(..., Executor);

First, Javers will take a snapshot of user's objects, it's fast so it can be done in the current thread.

Then, DB reads (loading latest snapshots) and DB writes (inserting new snapshots) can be done asynchronously (submitted to the given Executor).

As you mentioned, it requires the new approach to DB transactions. We plan to implement the Commit Withdrawal feature, so the app would be able to withdraw JaVers' commit after main DB rollback. See https://github.com/javers/javers/issues/588