Search code examples
spring-batchspring-java-config

Spring Batch - How to prevent batch from storing transactions in DB


First the problem statement: I am using Spring-Batch in my DEV environment fine. When I move the code to a production environment I am running into a problem. In my DEV environment, Spring-Batch is able to create it's transaction data tables in our DB2 database server with out problem. This is not a option when we go to PROD as this is a read only job.

Attempted solution:

Search Stack Overflow I found this posting: Spring-Batch without persisting metadata to database?

Which sounded perfect, so I added

@Bean
public ResourcelessTransactionManager transactionManager() {
    return new ResourcelessTransactionManager();
}

@Bean
public JobRepository jobRepository(ResourcelessTransactionManager transactionManager) throws Exception {
    MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(transactionManager);
    mapJobRepositoryFactoryBean.setTransactionManager(transactionManager);

    return mapJobRepositoryFactoryBean.getObject();
}

I also added it to my Job by calling .reporitory(jobRepository).

But I get

Caused by: java.lang.NullPointerException: null
    at       org.springframework.batch.core.repository.dao.MapJobExecutionDao.synchronizeStatus(MapJobExecutionDao.java:158) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]

So I am not sure what to do here. I am new to Spring so I am teaching myself as I go. I am open to other solutions, such as an in memory database, but I have not been able to get them to work either. I do NOT need to save any state or session information between runs, but the data base query I am running will return around a million or so rows, so I will need to get that in chunks.

Any suggestions or help would be greatly appreciated.


Solution

  • The following seems to have done the job for me:

    @Bean
    public DataSource dataSource() {        
    
        EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
        EmbeddedDatabase db = builder
            .setType(EmbeddedDatabaseType.HSQL) 
            .build();
        return db;
    }
    

    Now Spring is not creating tables in our production database, and when the JVM exits state is lost so nothing seems to be hanging around.

    UPDATE: The above code has caused concurrency errors for us. We have addressed this by abandoning the EmbeddedDatabaseBuilder and declaring the HSQLDB this way instead:

    @Bean
        public BasicDataSource dataSource() {
            BasicDataSource dataSource = new BasicDataSource();
              dataSource.setDriverClassName("org.hsqldb.jdbcDriver");
              dataSource.setUrl("jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc");
              dataSource.setUsername("sa");
              dataSource.setPassword("");
            return dataSource;
        }   
    

    The primary difference is that we are able to specify mvcc (Multiversion concurrency control) in connection string which resolves the issue.