Currently I have a setup like below. On running the batch job locally the job will create the necessary metadata tables automatically using the data-source
property values since initialize-schema
is set to always. Liquibase will also run and create any tables listed in its changelog.
Here is my application.yml
file
spring:
batch:
initialize-schema: always
job:
enabled: true
liquibase:
url: db_url
user: deploy_user
password: deploy_pass
change-log: classpath:db/changelog/db.changelog-master.yaml
enabled: true
data-source:
mysql:
user: r_user
password: r_pass
jdbc-url: db_url
Here is my db.changelog-master.yaml
file.
databaseChangeLog:
- changeSet:
dbms: mysql
id: create-sample-table
author: me
sql: CREATE TABLE sample_table (
sample_id VARCHAR(255) NOT NULL,
sample_text TEXT,
PRIMARY KEY (samoke_id)
) ENGINE=InnoDB DEFAULT
CHARSET=utf8 COLLATE=utf8_bin;
Mysql datasource config:
@Configuration
public class DataSourceConfiguration {
@Primary
@Bean(name = "mySQLDataSource")
@ConfigurationProperties("data-source.mysql")
public DataSource mySQLDataSource() {
return DataSourceBuilder.create().type(HikariDataSource.class).build();
}
}
Liquibase Configuration (probably posting more than what's needed):
@Configuration
@EnableConfigurationProperties(LiquibaseProperties.class)
public class LiquibaseConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(LiquibaseConfiguration.class);
@Autowired
private LiquibaseProperties liquibaseProperties;
public DataSource liquibaseDataSource() {
DataSourceBuilder factory = DataSourceBuilder
.create()
.url(liquibaseProperties.getUrl())
.username(liquibaseProperties.getUser())
.password(liquibaseProperties.getPassword());
return factory.build();
}
public void testLiquibaseConnection() throws SQLException {
LOG.info("Testing connection to Liquibase (in case PCF restarts and we have stale dynamic secrets)...");
liquibaseDataSource().getConnection();
LOG.info("Testing connection to Liquibase (in case PCF restarts and we have stale dynamic secrets)... Succeeded");
}
@Bean
public SpringLiquibase liquibase() {
try {
testLiquibaseConnection();
} catch (Exception ex) {
LOG.warn("WARNING: Could not connect to the database using " + liquibaseProperties.getUser() + ", so we will be skipping the Liquibase Migration for now. ", ex);
return null;
}
SpringLiquibase liquibase = new SpringLiquibase();
liquibase.setChangeLog(this.liquibaseProperties.getChangeLog());
liquibase.setContexts(this.liquibaseProperties.getContexts());
liquibase.setDataSource(liquibaseDataSource());
liquibase.setDefaultSchema(this.liquibaseProperties.getDefaultSchema());
liquibase.setDropFirst(this.liquibaseProperties.isDropFirst());
liquibase.setShouldRun(this.liquibaseProperties.isEnabled());
liquibase.setLabels(this.liquibaseProperties.getLabels());
liquibase.setChangeLogParameters(this.liquibaseProperties.getParameters());
return liquibase;
}
}
The issue is we have different credentials for creating/deploying tables and reading/writing to tables in our deployed environments. So the below setup will work to create tables via Liquibase, but fail creating the metadata tables due to having the incorrect credentials upon deployment. Our current work-around to get the metadata tables created is to deploy with the data-source
properties having deploy credentials, run the job to initialize the tables and then redeploy with read/write credentials. (We can't just leave the deploy credentials for reads because they have very short TTL).
Is it possible to create the metadata tables for Spring Batch via Liquibase automatically? Specifically, without adding the creation SQL manually to the changelog files?
UPDATE:
Using veljkost's answer below having a changelog file that looks like this works:
databaseChangeLog:
- changeSet:
dbms: mysql
id: create-spring-batch-metadata
author: dev.me
changes:
- sqlFile:
encoding: UTF-8
path: classpath:/org/springframework/batch/core/schema-mysql.sql
relativeToChangelogFile: false
splitStatements: true
stripComments: true
Yes, you can reference the schema files that already exist in Spring Batch project. In org.springframework.batch.core
package you can find schema-*.sql files where * is the name of the targeted db. Since you are running on mysql, your change set would look something like this:
- changeSet:
id: 1234
author: adam.sandler
changes:
- sqlFile:
encoding: utf8
path: classpath:/org/springframework/batch/core/schema-mysql.sql
relativeToChangelogFile: false
splitStatements: true
stripComments: true