Spring Batch Push to Queue after DB Slave Step Writer Commit

Question

Our requirement is to send the Items to queue after Slave steps DB writer Commit.

This is a Local partition application that reads files and split into 30 Slave steps and insert into one table. After the table insert commit we want few of the attributes to send from slave writer to the embedded queue that will do another insert into another table(This is part of another transaction) . We want to make sure queue transaction commit should happen after spring batch slave writer commit

What we've tried

We have tried ItemWriteListener@AfterWrite. This doesn't help as its before committing the transaction. Planning to use ChunkListener@afterChunk but this doesn't have items chunk and we need to set it from slave step threads and not sure any impact on concurrency. Please let us know any other best alternatives

Slave Step for Reference

 @Bean
    public Step slaveStep(ItemReader<BaseDTO> dispatchReader, CompositeItemProcessor<BaseDTO, BaseDTO> dispatchProcessor, CompositeItemWriter<BaseDTO> dispatchWriter) throws Exception {
        return new StepBuilder("slaveStep", jobRepository).<BaseDTO, BaseDTO>chunk(chunkSize, dispatchTransactionManager).reader(dispatchReader)
                .processor(dispatchProcessor).writer(dispatchWriter)
                .faultTolerant().skipLimit(skipErrorCount)
                .skip(Exception.class)
                .noRetry(Exception.class)
                .noRollback(Exception.class)
                .processorNonTransactional()
                .listener(itemSkipListener())
                .listener(stepExecutionListener())
                .build();
    }


 

 

Answer 1

I would keep it simple (compared to using listeners or composite writers and dealing with XA transactions etc) and add a subsequent step that reads the committed items from table and sends them to the queue. This step would start after the partitioned step has finished writing data into the table.