Search code examples
spring-batch

Some questions about multithreaded steps


We use a TaskExecutor in our step and use StepScope reader, processor and writer. We set page size to 15000, chunk size to 50, and thread count to 20.

Firstly, does each thread get its own instance of StepScope reader, processor and writer or there is only instance of reader,processor,writer per step and shared across all threads?

Also, is the following correct?

  1. The reader will read up to pageSize items (e.g. 15000) or finish the step if none.
  2. Those 15000 will be broken into chunks (e.g. 50 items/chunk) to be distributed among the 20 threads.
  3. Each thread will call the processor 50 times to process the chunk items sequentially.
  4. Each thread will call the writer 50 times but all within a single transaction.
  5. Steps 3 and 4 will repeat until all 15000 items are processed and written.
  6. Reader will go to step 1.

Thanks.


Solution

  • Firstly, does each thread get its own instance of StepScope reader, processor and writer or there is only instance of reader,processor,writer per step and shared across all threads?

    Step-scoped beans are not shared, each step will have its own instance of the bean.

    Also, is the following correct?

    No, in a multi-threaded step, each chunk is processed by a separate thread from the task executor. Therefore, the order of items reading and processing is undefined (compared to a sequential execution).

    Please refer to the Multi-threaded Step section for more details.