I am using Spring Batch for the first time, and originally I just timed a task that fetches data from a database and writes it to an Excel via ExcelWriter after some processing, and that worked.
However, when I try to realize: each time the result of the timed task processing is corresponding to the generation of an Excel this does not work. When I printed the logs, I noticed that the ExcelWriter used by the timed tasks seemed to be the same, even though I recreated the ExcelWriter instance once for each timed task.
Finally, my question is: how to solve this problem: always calling the same ExcelWriter.
This is my core code:
Let me know if any core code is missing, here are all the file names:
BatchConfig.java
BatchJobCompletionListener.java
BatchScheduler.java
CustomSkipListener.java
ExcelWriter.java
JobParametersHolder.java
Transaction.java
TransactionProcessor.java
TransactionQueryConfig.java
TransactionRowMapper.java
@Configuration
@EnableBatchProcessing
public class BatchConfig {
private final DataSource dataSource;
private final JobBuilderFactory jobBuilderFactory;
private final StepBuilderFactory stepBuilderFactory;
public BatchConfig(DataSource dataSource, JobBuilderFactory jobBuilderFactory, StepBuilderFactory stepBuilderFactory) {
this.dataSource = dataSource;
this.jobBuilderFactory = jobBuilderFactory;
this.stepBuilderFactory = stepBuilderFactory;
}
@Bean
public Job transactionExportJob(Step exportStep, BatchJobCompletionListener listener) {
return jobBuilderFactory.get("transactionExportJob")
.incrementer(new RunIdIncrementer())
.start(exportStep)
.listener(listener)
.build();
}
@Bean
public Step exportStep(ItemReader<Transaction> reader, ExcelWriter excelWriter, PlatformTransactionManager transactionManager) {
return stepBuilderFactory.get("exportStep")
.<Transaction, Transaction>chunk(5000)
.reader(reader)
.processor(new TransactionProcessor()) // handle data
.writer(excelWriter)
.faultTolerant() // enabling fault tolerance
.retryLimit(3)
.retry(Exception.class)
.skip(Exception.class)
.skipLimit(100)
.listener(new CustomSkipListener()) // listing skip
.transactionManager(transactionManager)
.build();
}
@Bean
public JdbcPagingItemReader<Transaction> reader(PagingQueryProvider transactionQueryProvider) {
JdbcPagingItemReader<Transaction> reader = new JdbcPagingItemReader<>();
reader.setDataSource(dataSource);
reader.setFetchSize(5000);
reader.setRowMapper(new TransactionRowMapper());
reader.setQueryProvider(transactionQueryProvider);
return reader;
}
}
@Component
public class ExcelWriter implements ItemWriter<Transaction> {
private final String filePath;
private final Workbook workbook;
private final Sheet sheet;
private int rowCount = 1;
public ExcelWriter() {
this.filePath = "transactions_" + UUID.randomUUID().toString() + ".xlsx";
this.workbook = new XSSFWorkbook();
this.sheet = workbook.createSheet("Transactions");
createHeaderRow();
}
@Override
public void write(List<? extends Transaction> items) throws Exception {
System.out.println("filePath: " + filePath);
for (Transaction transaction : items) {
Row row = sheet.createRow(rowCount++);
row.createCell(0).setCellValue(transaction.getUniqueTxId());
row.createCell(1).setCellValue(transaction.getCreatedTime().toString());
}
try (FileOutputStream out = new FileOutputStream(filePath)) {
workbook.write(out);
} catch (IOException e) {
System.err.println("Error closing Excel file: " + e.getMessage());
throw new IOException("Failed to close workbook", e);
}
}
private void createHeaderRow() {
Row header = sheet.createRow(0);
header.createCell(0).setCellValue("Unique Tx Id");
header.createCell(1).setCellValue("Created Time");
}
public void closeWorkbook() throws IOException {
workbook.close();
}
}
@Component
@EnableScheduling
public class BatchScheduler {
private final JobLauncher jobLauncher;
private final Job transactionExportJob;
private final JobParametersHolder jobParametersHolder;
public BatchScheduler(JobLauncher jobLauncher, Job transactionExportJob, JobParametersHolder jobParametersHolder) {
this.jobLauncher = jobLauncher;
this.transactionExportJob = transactionExportJob;
this.jobParametersHolder = jobParametersHolder;
}
@Scheduled(fixedRate = 60000, initialDelay = 60000, zone = "Asia/Shanghai")
public void runBatchJob() {
try {
ExcelWriter excelWriter = new ExcelWriter();
LocalDateTime startTime = LocalDateTime.now().minusDays(1).withHour(22);
LocalDateTime endTime = LocalDateTime.now().withHour(22);
JobParameters jobParameters = new JobParametersBuilder()
.addDate("startTime", Date.from(startTime.atZone(ZoneId.of("Asia/Shanghai")).toInstant()))
.addDate("endTime", Date.from(endTime.atZone(ZoneId.of("Asia/Shanghai")).toInstant()))
.toJobParameters();
System.out.println("jobParameters: " + jobParameters);
jobParametersHolder.setJobParameters(jobParameters);
System.out.println("Batch Job started");
JobExecution execution = jobLauncher.run(transactionExportJob, jobParameters);
System.out.println("Batch job completed with status: {}" + execution.getStatus());
excelWriter.closeWorkbook();
} catch (Exception e) {
System.out.println("Batch job execution failed: {}" + e.getMessage() + e);
}
}
}
----- Edit Content -----
Thanks to @lance-java, after I removed the @Component annotation from ExcelWriter, using the @JobScope annotation to identify ExcelWriter would cause a corresponding ExcelWriter instance to be recreated each time the Job is run, thus achieving what I was trying to achieve.
The @Component annotation on ExcelWriter defines a bean in spring's application context
@Component
public class ExcelWriter implements ItemWriter<Transaction> {
...
}
You inject the ExcelWriter instance (created above) into the "exportStep" bean
@Configuration
@EnableBatchProcessing
public class BatchConfig {
...
@Bean
public Step exportStep(..., ExcelWriter excelWriter, ...) {
...
}
...
}
The ExcelWriter instance created in BatchScheduler.runBatchJob() is irrelevant, this is not used anywhere
@Component
@EnableScheduling
public class BatchScheduler {
@Scheduled(fixedRate = 60000, initialDelay = 60000, zone = "Asia/Shanghai")
public void runBatchJob() {
try {
ExcelWriter excelWriter = new ExcelWriter();
...
excelWriter.closeWorkbook();
}
...
}
}