spring batch: required a bean named 'dataSource' that could not be found

Question

I am upgrading spring boot from version 2.6.6 to 3.1.3.

I did the required JobRepository related modifications by referring this sof post.

Everything seems to be going well but then I am facing below error -

***************************
APPLICATION FAILED TO START
***************************

Description:

Parameter 1 of method stepOne in com.mycompany.retail.esl.data_manager_job.product_publisher_job.ProductPublisherJob required a bean named 'dataSource' that could not be found.

The injection point has the following annotations:
    - @org.springframework.beans.factory.annotation.Qualifier("PRODUCT_PUBLISHER_JOB_STEP_ONE")


Action:

Consider defining a bean named 'dataSource' in your configuration.

My step builder and job builder code is as below -

@Configuration
@EnableBatchProcessing
@RequiredArgsConstructor
public class ProductPublisherJob {

  private static final int DEFAULT_CHUNK_SIZE = 25;
  public static final String JOB_NAME = JobName.PRODUCT_PUBLISHER_JOB;

  @Bean(ProductPublisherJob.JOB_NAME)
  public Job job(
      @Qualifier(ProductPublisherJob.JOB_NAME + "_STEP_ONE") Step stepOne,
      JobRepository jobRepository,
      JobExecutionListener jobExecutionListener,
      @Qualifier("ArticleFlatFilePublisher") ArticleFlatFilePublisher articleFlatFilePublisher
  ) {
    return new JobBuilder(ProductPublisherJob.JOB_NAME, jobRepository)
        .incrementer(new RunIdIncrementer())
        .flow(stepOne)
        .end()
        .listener(jobExecutionListener)
        .listener(articleFlatFilePublisher)
        .build();
  }

  @Bean(ProductPublisherJob.JOB_NAME + "_STEP_ONE")
  public Step stepOne(
      @Qualifier("ProductPublisherJobPageReader") ProductPublisherJobPageReader pageReader,
      JobRepository jobRepository,
      @Qualifier("ProductItemProcessor") ProductItemProcessor itemProcessor,
      @Qualifier("ArticleFlatFileWriter") ArticleFlatFileWriter itemWriter
  ) {
    return new StepBuilder(ProductPublisherJob.JOB_NAME + "_ONE", jobRepository)
        .<Product, Article>chunk(DEFAULT_CHUNK_SIZE)
        .reader(pageReader)
        .processor(itemProcessor)
        .writer(itemWriter)
        .build();
  }
}

I do have the required datasource props defined in application.properties and further it was all working well in spring boot 2.6.6.
Any suggestions on how to fix this. Any help will be greatly appreciated as this is a prod blocker for me.

EDIT 1 - regrading the required data source bean

I have below in application.properties

spring.batch.datasource.driver-class-name = org.h2.Driver
spring.batch.datasource.password = sa
spring.batch.datasource.url = jdbc:h2:mem:db;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=false;
spring.batch.datasource.username = sa

and in build.gradle i have -

implementation "com.h2database:h2:2.2.224"
implementation "org.springframework.boot:spring-boot-starter-batch"
implementation "org.springframework.boot:spring-boot-starter-data-jpa"

IS THERE ANYTHING MORE I HAVE TO DO TO GET THE DATASOURCE BEAN IN TO APPLICATION CONTEXT.

EDIT 2 - added datasource bean explicitly.
I explicitly added below bean definition in to the configuration class.

@Bean
public DataSource dataSource() {
  DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create(
  dataSourceBuilder.driverClassName("org.h2.Driver");
  dataSourceBuilder.url("jdbc:h2:mem:test");
  dataSourceBuilder.username("SA");
  dataSourceBuilder.password("");
  return dataSourceBuilder.build();
}

It suggested:

Consider defining a bean named 'transactionManager' in your configuration.

so I added -

@Bean
public PlatformTransactionManager transactionManager() {
  return new DataSourceTransactionManager(dataSource());
}

But now it's giving another error still related to transactionManager -

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'PRODUCT_PUBLISHER_JOB' defined in class path resource [com/wayfair/retail/esl/data_manager_job/product_publisher_job/ProductPublisherJob.class]: Unsatisfied dependency expressed through method 'job' parameter 0: Error creating bean with name 'PRODUCT_PUBLISHER_JOB_STEP_ONE' defined in class path resource [com/wayfair/retail/esl/data_manager_job/product_publisher_job/ProductPublisherJob.class]: Failed to instantiate [org.springframework.batch.core.Step]: Factory method 'stepOne' threw exception with message: java.lang.IllegalStateException: A transaction manager must be provided

I am providing transaction manager explicitly, still why does it complain.

Edit 3 - Itemwriter code as requested by Gurkan.

@Component("ArticleFlatFileWriter")
public class ArticleFlatFileWriter extends FlatFileItemWriter<Article> {

  private final ArticleMapper articleMapper;
  ArticleFlatFileWriter(
      @Value("${app.storage.localFile}") String file,
      @Value("${app.storage.file.csv.header}") Boolean includeHeader,
      @Qualifier("ArticleLineAggregator") ArticleLineAggregator lineAggregator,
      ArticleMapper articleMapper
  ) {
    super();

    if (Boolean.TRUE.equals(includeHeader)) {
      this.setHeaderCallback(this::headerCallback);
    }

    var fileSystemResource = new FileSystemResource(file);

    this.setAppendAllowed(true);
    this.setEncoding("UTF-8");
    this.setLineAggregator(lineAggregator);
    this.setResource(fileSystemResource);
    this.setShouldDeleteIfExists(true);
    this.articleMapper = articleMapper;
  }

  private void headerCallback(Writer writer) throws IOException {
    writer.write(String.join(ArticleLineTokenizer.DEFAULT_DELIMITER, this.articleMapper.getHeaderNames()));
  }

  protected boolean isAppendAllowed() {
    return this.append;
  }
}

Edit 4 - After updating the StepBuilder code to below (like also suggested by skillup), Transaction manager issue is solved but am facing Error creating bean with name 'jpaMappingContext': JPA metamodel must not be empty. I have posted a new question for this.

@Bean(UpdateAssignedProductsJob.JOB_NAME + "_STEP_ONE")
public Step stepOne(
    @Qualifier("UpdateAssignedProductsJobPagedItemReader") UpdateAssignedProductsJobPagedItemReader itemReader,
    JobRepository jobRepository, PlatformTransactionManager transactionManager,
    @Qualifier("ProductItemProcessor") ProductItemProcessor itemProcessor,
    @Qualifier("ArticleFlatFileWriter") ArticleFlatFileWriter itemWriter
) {
   return new StepBuilder(UpdateAssignedProductsJob.JOB_NAME + "_STEP_ONE", jobRepository)
      .<Product, Article>chunk(DEFAULT_CHUNK_SIZE, transactionManager)
      .reader(itemReader)
      .processor(itemProcessor)
      .writer(itemWriter)
      .build();
}

Answer 1

You should replace the code of your step like this :

  @Bean(ProductPublisherJob.JOB_NAME + "_STEP_ONE")
  public Step stepOne( JobRepository jobRepository,PlatformTransactionManager transactionManager,
      @Qualifier("ProductPublisherJobPageReader") ProductPublisherJobPageReader pageReader,
      @Qualifier("ProductItemProcessor") ProductItemProcessor itemProcessor,
      @Qualifier("ArticleFlatFileWriter") ArticleFlatFileWriter itemWriter
  ) {
    return new StepBuilder(ProductPublisherJob.JOB_NAME + "_ONE", jobRepository)
        .<Product, Article>chunk(DEFAULT_CHUNK_SIZE, transactionManager)
        .reader(pageReader)
        .processor(itemProcessor)
        .writer(itemWriter)
        .build();
  }