I am curious to know if anyone has been able to successfully use a cache such Hazelcast or Redis with spring batch as the persistence layer instead of a full RDMS.
The reason I am asking such a question is I would like to measure the performance of my application without the latency induced from transacting when many parallel steps are processing as well as when there is a high amount of threads processing in a multi-threaded step.
My current thoughts on this is that it is possible as we can use the DataSource configuration with a JobRepository.
If this is not possible is there a way to do the step level transactions on the persistence layer(PostgreSQL for example) asynchronously to avoid the seconds delay in a step starting and ending.
Thank you if any code snippet is needed I will be more than happy to share.
You can use any persistence technology you want for the job repository, as long as your implementation respects the JobRepository contract.
Now if the requirement is to measure the performance of your job under particular conditions, I would not change the persistence layer just for that. I would rather recommend to profile the execution with the same job repository in a staging environment that is as close as possible to your production environment. Any decent profiler would give you the performance details of your step and the overhead of transactions.