Wait for postgres service to start in dagger

Question

I am using dagger wherein I am writing code using golang sdk. I want to perform integration testing for which I would need to start a postgres service. Once the postgres service is up I would want to start liquibase service to migrate data into the postgres database.

Here is my code

func (m *DaggerEnvy) IntegrationTest(ctx context.Context, source *dagger.Directory,
    npmrcFile *dagger.Secret,
    // optional
    // +default="postgres"
    dbName,
    // optional
    // +default="postgres"
    dbUser,
    // optional
    // +default="postgres"
    dbPassword string,
) (string, error) {

    // postgres service
    postgresService := m.postgresService(ctx, dbName, dbUser, dbPassword)
    postgresService.Start(ctx)
    defer postgresService.Stop(ctx)

    // liquibase service
    liquibaseService := m.liquibaseContainer(ctx, source, postgresService, dbName)
    liquibaseService.Start(ctx)
    defer liquibaseService.Stop(ctx)

    return m.buildNodeEnv(source, npmrcFile).
        WithServiceBinding("postgres-service", postgresService).
        WithServiceBinding("liquibase-service", liquibaseService).
        WithExec([]string{"npm", "run", "migrate:db"}).
        Stderr(ctx)
}


func (m *DaggerEnvy) postgresService(
    ctx context.Context,
    // optional
    // +default="postgres"
    dbName,
    // optional
    // +default="root"
    dbUser,
    // optional
    // +default="postgres"
    dbPassword string,
) *dagger.Service {
    // ? should optional parameters be stored in .env file?
    // postgresDataVolume := dag.CacheVolume("./postgres-data") // Path for local persistence

    return dag.Container().From("postgres:17").
        WithEnvVariable(constants.POSTGRES_USER, dbUser).
        WithEnvVariable(constants.POSTGRES_PASSWORD, dbPassword).
        WithEnvVariable(constants.POSTGRES_DB, dbName).
        WithExposedPort(5432).
        // WithMountedCache("/var/lib/postgresql/data", postgresDataVolume).
        AsService(dagger.ContainerAsServiceOpts{
            UseEntrypoint: true,
        })
}

func (m *DaggerEnvy) liquibaseContainer(
    ctx context.Context,
    source *dagger.Directory,
    postgresService *dagger.Service,
    // +optional
    // +default="postgres"
    dbName string,
) *dagger.Service {
    return dag.Container().
        From("liquibase").
        WithFile("/liquibase/master.yaml", source.File("src/server/schema/liquibase/master.yaml")).
        WithDirectory("/liquibase/changelog", source.Directory("src/server/schema/changelog")).
        WithEnvVariable("LIQUIBASE_COMMAND_URL", fmt.Sprintf("jdbc:postgresql://postgres-service:5432/%s", dbName)).
        WithEnvVariable("LIQUIBASE_COMMAND_USERNAME", "postgres").
        WithEnvVariable("LIQUIBASE_COMMAND_PASSWORD", "postgres").
        WithEnvVariable("LIQUIBASE_COMMAND_CHANGELOG_FILE", "master.yaml").
        WithExec([]string{"liquibase", "update"}).
        AsService()
}

When I run this using dagger call integration-test ... then I am getting following error

Starting Liquibase at 12:14:08 using Java 17.0.13 (version 4.30.0 #4943 buil    
 2024-10-31 17:00+0000)                                                         
Liquibase Version: 4.30.0                                                       
Liquibase Open Source 4.30.0 by Liquibase                                       
ERROR: Exception Details                                                        
ERROR: Exception Primary Class:  UnknownHostException                           
ERROR: Exception Primary Reason:  postgres-service                              
ERROR: Exception Primary Source:  4.30.0                                        
                                                                                
Unexpected error running Liquibase: Connection could not be created to jdbc:    
postgresql://postgres-service:5432/postgres with driver org.postgresql.Drive    
r.  The connection attempt failed.                                              
                                                                                
For more information, please use the --log-level flag                           
! process "liquibase update" did not complete successfully: exit code: 1

According to my understanding the liquibase-service has started its execution before postgres-service is ready to accept connections. I am also facing a similar issue where is WithExec([]string{"npm", "run", "migrate:db"}) gets executed before completion of both services.

How should I make the service binding run in synchronously? I am using dagger version: v0.15.1

I have generated dagger files using following command: dagger init --sdk=go

Let me know if anything more is required here.

Answer 1

We dont need to manually wait for the postgres service to start and accept connections. When we use it along with container using WithServiceBinding("postgres-service", postgresService) it should work correctly. If we want to start the service explicitly then we can use Start method

postgresService := m.postgresService(ctx, dbName, dbUser, dbPassword)
postgresService, err := postgresService.Start(ctx)
if err != nil {
  return err
}

This should start the service before binding it with the container