Search code examples
cloudtaskspring-batchpartitioning

Spring Batch Remote partitioning not working


We have a Spring Batch application using Spring Boot 2.5.2 in which the batch jobs are deployed as Cloud tasks. While using remote partitioning, we noticed that the entire partition pay-load gets sent to all the worker JVM's.

For example, if our payload is 1000 records with a partition-size of 500, then both the workers get all the 1000 records each instead of 500 each. The execution context seems to have the division done correctly in the Master as it is printing partition0 with the first 500 records and partition1 with the next 500. So we are surprised why Spring Batch is not passing the payloads accordingly to both the workers.

Any help on this would be appreciated.


Solution

  • Got it to work with Spring Boot 2.1.1 and corresponding lower versions of Cloud, Deployer and Cloud task.