testcontainers, mariadb, shardingsphere and flyway

Question

I'm looking for some really good examples that can inspire me how to setup my integration test given a setup with Testcontainers, mariadb, shardingsphere and flyway.

Wanting 3 shards, I assume I'll need to init 3 different datasources, one for each shard like the following

spring:
  config:
    activate:
      on-profile: test
  shardingsphere:
    datasource:
      names: test-api-1, test-api-2, test-api-3,
      test-api-1:
        type: org.apache.commons.dbcp2.BasicDataSource
        driver-class-name: org.testcontainers.jdbc.ContainerDatabaseDriver
        url: jdbc:tc:mariadb:10.6.7:///
        validationQuery: /* ping */ select 1
        name: testapi01
        username: root
        password: root1234
      test-api-2:
        type: org.apache.commons.dbcp2.BasicDataSource
        driver-class-name: org.testcontainers.jdbc.ContainerDatabaseDriver
        url: jdbc:tc:mariadb:10.6.7:///
        validationQuery: /* ping */ select 1
        name: testapi02
        username: root
        password: root1234
      test-api-3:
        type: org.apache.commons.dbcp2.BasicDataSource
        driver-class-name: org.testcontainers.jdbc.ContainerDatabaseDriver
        url: jdbc:tc:mariadb:10.6.7:///
        validationQuery: /* ping */ select 1
        name: testapi03
        username: root
        password: root1234

What I can't figure out is how to configure flyway properly. So bottom line, I'm looking for a couple of good practical examples that can point me in the right direction. Also feel free to correct me if my above assumptions are wrong.

So far we've been running integration tests that are based on a single datasource as the main focus has been on testing with a production like database migration. But as we're experimenting with changing our shard selection we need to test with multiple shards as well.

Answer 1

Actually, there are two methods,

First, if you use shardingsphere-jdbc with flyway, it's possible to configure them separately. That means here you use 3 DBs for sharding and need to tell flyway all these 3 DBs and all the sharding actual tables. Notice, this way will cause the drift issue since shardingsphere-jdbc can not be aware of changes from the flyway.

Second, use shardingsphere-proxy to do sharding and flyway for the migration. Because shardingsphere-proxy shares the same functions with shardingsphere-jdbc and works as a database server. Therefore, you can create a logic-sharding database with many logic-sharding tables by ShardingSphere-Proxy [1]. Then tell flyway with this logic DB and sharding tables, username and password to do the migration.

flyway.user=shardingsphere-proxy-username # Defined in a server.yaml file.
flyway.password=shardingsphere-proxy-databasePassword # Defined in a server.yaml file.
flyway.schemas=shardingsphere-proxy-schemaName # Defined in a config-sharding.yaml file.

[1] https://shardingsphere.apache.org/document/5.2.1/en/user-manual/shardingsphere-proxy/startup/