How to persist data record by record in a Spring Batch Tasklet Job?

Question

I’m developing a Spring Batch Tasklet Job that performs the following steps:

1. Fetches files from a folder.
2. Processes the file records.
3. Validates each record.
4. Persists the data into the database.

The issue I’m facing is that the data persistence only happens after the entire Tasklet execution is complete. I need each record to be persisted immediately after it is processed and validated, but I haven’t been able to achieve this.

Here’s what I’ve tried so far:

• Adjusting the transaction scope of the persistence method, but the data still waits until the Tasklet finishes to be saved.
• Using a chunk-based Job instead of a Tasklet, but I couldn’t adapt it to my current scenario.
• Various combinations of Spring Batch transaction configurations, with no success. My priority is to ensure that, if the Job fails, the already processed records are committed to the database without depending on the complete Tasklet execution.

Is there a way to work around this issue and allow each record to be immediately persisted into the database during the Tasklet execution?

Any suggestions on configurations or approaches would be highly appreciated!

Answer 1

Manually handling the PlatformTransactionManager should do the job.

@Component
public class CustomTasklet implements Tasklet {

    @Autowired
    private PlatformTransactionManager transactionManager;

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) {
        TransactionStatus status = transactionManager.getTransaction(new DefaultTransactionDefinition());

        try {
            // Do some writes here

            transactionManager.commit(status);
        } catch (Exception e) {
            transactionManager.rollback(status);
        }

        return RepeatStatus.FINISHED;
    }
}

Although I would highly suggest splitting your task into the classic read-process-write steps.