Spring Quartz Scheduler not working in ECS Cluster

Question

I am working on a scheduling application using Spring Boot(2.2.7.RELEASE), Quartz Scheduler(2.3.2). Application deployed in AWS ECS Cluster and running in multiple EC2 instances based on the load. I am using the AWS RDS Mysql database and created required quartz tables. I have used the following properties file for quartz.

quartz.properties-

org.quartz.scheduler.instanceName = MyClusteredScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 25
org.quartz.threadPool.threadPriority = 5
org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties = true
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 20000

Application Starting log -

2020-10-13 23:33:02.853  INFO 938 --- [           main] org.quartz.core.QuartzScheduler          : Quartz Scheduler v.2.3.2 created.
2020-10-13 23:33:02.856  INFO 938 --- [           main] o.s.s.quartz.LocalDataSourceJobStore     : Using db table-based data access locking (synchronization).
2020-10-13 23:33:02.858  INFO 938 --- [           main] o.s.s.quartz.LocalDataSourceJobStore     : JobStoreCMT initialized.
2020-10-13 23:33:02.859  INFO 938 --- [           main] org.quartz.core.QuartzScheduler          : Scheduler meta-data: Quartz Scheduler (v2.3.2) 'quartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.springframework.scheduling.quartz.LocalDataSourceJobStore' 
- which supports persistence. and is not clustered.

Problem is multiple EC2 instances are picking up the same job at same time. Because of that getting the wrong result.

Can somebody please let me know what I am missing here? What I need to do here to make it work in the ECS cluster. Every time when the Spring application is getting started getting, it's showing the log message is not clustered.

Answer 1

This's been resolved. I have added the following configuration details to the application.properties file and removed the quartz.properties file

spring.quartz.job-store-type = jdbc
spring.quartz.properties.org.quartz.scheduler.instanceName = MyClusteredScheduler
spring.quartz.properties.org.quartz.jobStore.isClustered = true
spring.quartz.properties.org.quartz.scheduler.instanceId = AUTO
spring.quartz.properties.org.quartz.jobStore.useProperties = true
spring.quartz.properties.org.quartz.jobStore.tablePrefix = QRTZ_
spring.quartz.properties.org.quartz.jobStore.isClustered = true
spring.quartz.properties.org.quartz.jobStore.clusterCheckinInterval = 20000

Now the batch job is working fine in ECS and at a time only one job is running in the ECS cluster.