Search code examples
spring-batch

How do I pass previous step data to partitioner


I am trying to run a partition oriented job and having trouble in accessing the stepExecutionContext stored data. Here is my job definition

<batch:job id="job1" restartable="false" incrementer="idIncrementer">
    <batch:step id="readwritestep" next="partitionStep">
        <batch:tasklet transaction-manager="transactionManager">
            <batch:chunk reader="reader1"
                         writer="writer1"
                         commit-interval="500"/>
        </batch:tasklet>
        <batch:listeners>
            <batch:listener ref="promotionListener"/>
        </batch:listeners>
    </batch:step>
    <batch:step id="partitionStep" >
        <batch:partition step="detailsStep" partitioner="partitioner">
            <batch:handler grid-size="10" task-executor="taskExecutor" />
        </batch:partition>
    </batch:step>
</batch:job>
<batch:step id="detailsStep">
    <batch:tasklet transaction-manager="transactionManager">
        <batch:chunk reader="reader2"
                         processor="processor"
                         writer="writer2"
                         commit-interval="1500"/>
    </batch:tasklet>
</batch:step>

While processing readwritestep I am storing some data in the step context and promoting to job context so that I can access in partioner. But custom partioner which I have implemented doesnt have any reference to the parent step where I can get access to the stored data...Even though partitioner is bound to STEP it cant access parent step data...am I missing something here? One option partitioner is giving is jdbctemplate to generate splitter contexts which I am not in intrest. I have tried to inject the @beforestep annotation to get access the context data but its not getting invoked..I dont want to perform the JDBC read to generate the slave data...I want to get the LIST data stored in the step/job context execution and generate the splitter contexts...Can somebody help me out to point me to the right direction so that I can access that data...

Here is the partitioning class...

     public class ProductDetailsPartitioner implements Partitioner {

     private List<Product> prds;


      @Override
      public Map<String, ExecutionContext> partition(int gridSize) {
       List<String> referencIds = new ArrayList<String>();
       for (Product prd : prds) {
        referencIds.add(prd.getReferenceId());
       }
       Map<String, ExecutionContext> results = new LinkedHashMap<String,ExecutionContext>();
      for (String referencId : referencIds) {
        ExecutionContext context = new ExecutionContext();
        context.put("referenceId", referencId);
        results.put("partition." + referencId, context);
      }
        return results;
    }

     @BeforeStep
     public void retrieveInterstepData(StepExecution stepExecution) {
           System.out.println("Entered Before step in partion");
           JobExecution jobExecution = stepExecution.getJobExecution();
           ExecutionContext jobContext = jobExecution.getExecutionContext();
           System.out.println("ExecutionContext"+jobContext);
           this.prds = (List<Product>) jobContext.get("products");

     }
  }

Solution

  • OK, I was able to pass this issue by taking another approach. Now I am processing partioner based on jobid rather than passing the reference through the execution context. Its running fine. Its hard to explain the real solution as it is based on the real bussiness needs.