I'm trying to implement a spring batch application that can read from two tables in mhysql database (using join query) and print the output on console and which can be scheduled to run in an interval say 10 seconds. Although my application is running without any error and also printing the steps in the console but it is not printing any data from the database.
Following is my pom.xml file-
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.2.5</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>com.rcg.m360</groupId>
<artifactId>crm-scheduler</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>m360-crm-scheduler</name>
<description>Project for the scheduler to get individual ids from Manifest
DB</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<version>3.3.0</version>
</dependency>
<dependency>
<groupId>com.mysql</groupId>
<artifactId>mysql-connector-j</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<!--
https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-test -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<version>3.2.5</version>
<scope>test</scope>
</dependency>
<!--
https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
Following is my application.yml file:
spring:
application:
name: m360-crm-scheduler
batch:
job:
enabled: true
jpa:
database-platform: org.hibernate.dialect.MySQLDialect
show-sql: true
hibernate:
ddl-auto: update
naming:
physical-strategy: org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
properties:
hibernate:
dialect: org.hibernate.dialect.MySQLDialect
format_sql: true
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: ${MANIFEST_DB_URL}
username: ${MANIFEST_DB_USERNAME}
password: ${MANIFEST_DB_PASSWORD}
initialize: true
server:
port: 8081
Here is the BatchConfiguration java class:
@Configuration
@EnableBatchProcessing
@EnableScheduling
public class BatchConfiguration {
@Autowired
private JobRepository jobRepository;
@Autowired
private PlatformTransactionManager transactionManager;
@Autowired
DataSourceConfig dataSourceConfig;
private static final Logger logger = LoggerFactory.getLogger(BatchConfiguration.class);
@Bean
public Job crmSchedulerJob() {
logger.info("Job execution started...");
return new JobBuilder("crmSchedulerJob", jobRepository)
.start(crmSchedulerStep())
.build();
}
@Bean
public Step crmSchedulerStep() {
return new StepBuilder("crmSchedulerStep", jobRepository)
.<SchedulerData, SchedulerData>chunk(10, transactionManager)
.reader(crmSchedulerReader())
.writer(crmSchedulerWriter())
.build();
}
public DataSource getDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(dataSourceConfig.getDriverClassName());
dataSource.setUrl(dataSourceConfig.getUrl());
dataSource.setUsername(dataSourceConfig.getUserName());
dataSource.setPassword(dataSourceConfig.getPassword());
return dataSource;
}
@Bean
public JdbcCursorItemReader<SchedulerData> crmSchedulerReader() {
String query = "select gRes.Individual_id, res.Ship_code, res.is_delete from Guest_reservation gRes join Reservation res \r\n"
+ "on gRes.Super_pnr_no=res.Super_pnr_no where res.Sailing_startdate>=DATE_SUB(CURDATE(), \r\n"
+ "INTERVAL 21 DAY)";
logger.info("The query is: "+query);
LocalDate date = LocalDate.now().minusDays(21);
return new JdbcCursorItemReaderBuilder<SchedulerData>()
.dataSource(getDataSource())
.name("crmSchedulerReader")
.sql(query)
.beanRowMapper(SchedulerData.class)
.build();
}
@Bean
public ItemWriter<SchedulerData> crmSchedulerWriter() {
logger.info("Start of writer method");
return items -> {
for (SchedulerData item : items) {
logger.info("Writing item: " + item);
}
};
}
@Scheduled(fixedRate = 10000)
public void perform() throws Exception {
logger.info("Start of perform method");
crmSchedulerJob();
}
}
Here is the main application class-
@SpringBootApplication
@EnableScheduling
public class M360CrmSchedulerApplication {
public static void main(String[] args) {
SpringApplication.run(M360CrmSchedulerApplication.class, args);
}
}
Here is the DataSourceConfig class -
@Data
@Component
public class DataSourceConfig {
@Value("${spring.datasource.url}")
private String url;
@Value("${spring.datasource.username}")
private String userName;
@Value("${spring.datasource.password}")
private String password;
@Value("${spring.datasource.driver-class-name}")
private String driverClassName;
}
Here is the DTO SchedulerData class:
public class SchedulerData {
private int individualId;
private String shipCode;
private String validityDateTo;
public SchedulerData(){
super();
}
public SchedulerData(int individualId, String shipCode, String validityDateTo) {
this.individualId = individualId;
this.shipCode = shipCode;
this.validityDateTo = validityDateTo;
}
public String getValidityDateTo() {
return validityDateTo;
}
public void setValidityDateTo(String validityDateTo) {
this.validityDateTo = validityDateTo;
}
public int getIndividualId() {
return individualId;
}
public void setIndividualId(int individualId) {
this.individualId = individualId;
}
public String getShipCode() {
return shipCode;
}
public void setShipCode(String shipCode) {
this.shipCode = shipCode;
}
}
I have specified the database connection details in the run configurations. I'm getting the following output in the console -
Started M360CrmSchedulerApplication in 3.549 seconds (process running for 4.115)
Start of perform method
Start of perform method
Start of perform method
From the log we can see that the perform method of the BatchConfiguration class is executing every 10 seconds but the job method is not executing as the logger statement in the job method is not printed. I'm not able to find why the control is not going to the job method, any help on this will be highly appreciated.
In your code is that the perform method is not actually triggering the execution of the Spring Batch job. Instead, it is merely calling the method that defines the job (crmSchedulerJob) without actually running it.
If you adapt your perform method like below then it should work.
@Autowired
private JobLauncher jobLauncher;
.
.
@Scheduled(fixedRate = 10000)
public void perform() throws Exception {
logger.info("Start of perform method");
JobParameters jobParameters = new JobParametersBuilder()
.addLong("time", System.currentTimeMillis())
.toJobParameters();
JobExecution jobExecution = jobLauncher.run(crmSchedulerJob(), jobParameters);
logger.info("Job Execution Status: " + jobExecution.getStatus());
}
Note: If you did not add table for Spring batch job then please do create those by executing this queries