Using Docker and Testcontainers for database integration testing

Question

After a bit of research I was led to think that Docker containers could be a good fit for database integration testing using test containers, as this would only require a Docker container running a database image reproducing the database schema against which the tests would be executed. This instance can either be run locally on each developer’s machine or, even better, one single instance can be shared by multiple developers: an automated test can be configured to start a database instance from the same image in Docker for every test method if needed using the @Rule annotation.

While trying to install Docker for Windows 7 to play with it, I kept getting the following error which seems to be related to VirtualBox:

docker Error creating machine: Error in driver during machine creation: Unable to start the VM: VBoxManage.exe startvm default --type headless failed:

Result code: E_FAIL (0x80004005)
Component: MachineWrap
Interface: IMachine

I will share how I worked around this problem to help other people who may have run into the same problem.

Answer 1

To workaround the issue described in the question, I followed the steps below:

1. Install VMware workstation player

2. Run a Linux Ubuntu virtual machine

Using iso file from Ubuntu website, run an Ubuntu appliance on VMware player

3. Install Docker on Ubuntu VM

$ sudo apt-get install docker.io

4. Post-install steps

4.1. Create the docker group and add your user

$ sudo groupadd docker

$ sudo usermod -aG docker $USER

4.2. Log out and log back in

so that your group membership is re-evaluated

4.3. Verify that you can run docker commands without sudo

$ docker run hello-world

5. Download database image

In this example it's Oracle 11g:

$ docker pull wnameless/oracle-xe-11g-r2

6. Run Database image

(if you plan to connect to the Oracle database instance from testcontainers, this step is not necessary)

$ docker run -d -p 49161:1521 wnameless/oracle-xe-11g-r2

For further details/options refer to docker hub:

7. Connect to the database

Using with following settings:

hostname: localhost

port: 49161

sid: xe

service name: xe

username: system

password: oracle

To connect from the host machine use the IP address of the VM instead of localhost . Run ifconfig on the guest Ubuntu VM to get the IP address

8. Configure Docker to be accessed remotely

Configure where the Docker daemon listens for connections (Required only if Docker needs to be accessed remotely, i.e. not from the guest Ubuntu system), as by default Docker daemon listens on unix sockets (local connections)

8.1. Create config file

Create /etc/systemd/system/docker.service.d/override.conf file with content to override default one (ExecStart=/usr/bin/dockerd -H fd://):

# /etc/systemd/system/docker.service.d/override.conf

[Service]

ExecStart=

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2376

To create this file you can run the following unix command:

printf "# /etc/systemd/system/docker.service.d/override.conf\n[Service]\nExecStart=\nExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2376" > /etc/systemd/system/docker.service.d/override.conf

8.2. Restart Docker

After saving this file, reload the configuration by running:

systemctl daemon-reload

Then restart Docker by running:

systemctl restart docker.service

8.3. Check your Docker daemon

After restarting docker service, you can see the port number in the output of either:

systemctl status docker.service

(You should see something like: /usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2376)

Or

sudo netstat -tunlp | grep dock

(You should see something like: tcp6 0 0 :::2376 :::* LISTEN 121686/dockerd)

8.4. Useful resources

How do I enable the remote API for dockerd

How to detect a docker daemon port

Configure where the docker daemon listens for connections

9. Set Docker Host environment variable

This step is necessary only if you plan to connect remotely to the database container using testcontainers API (e.g. from a Junit test) from the OS hosting the Ubuntu VM (docker dameon is running on the ubuntu VM)

Define environment variable: DOCKER_HOST = tcp://<Ubuntu machine's IP address>:2376 . Note the hostname is the ubuntu VM's. If this environment variable is not defined, testcontainers API (OracleContainer oracleContainer = new OracleContainer("wnameless/oracle-xe-11g");) will be expecting Docker daemon to be running on localhost (See code snippet further below)

10. Use the database container from a test class

Using testcontainer API, a Junit test can start a database instance from a the Docker image on Ubuntu's VM, execute queries against it and, eventually shut it down

Junit test class

package com.xxx.yyy.repository;
 
import static org.junit.Assert.assertEquals;
 
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.time.LocalDateTime;
import java.util.concurrent.TimeoutException;
 
import org.junit.ClassRule;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.TestInstance.Lifecycle;
import org.testcontainers.containers.OracleContainer;
 
@TestInstance(Lifecycle.PER_CLASS)
public class MyRepositoryIT {
 
    @ClassRule
    public OracleContainer oracleContainer;
 
    @BeforeAll
    public void setup() throws TimeoutException {
        String dockerHost = System.getenv("DOCKER_HOST");
        System.out.println("dockerHost: @" + dockerHost + "@");
 
        System.out.println("Starting Oracle Container... (" + LocalDateTime.now() + ")");
        oracleContainer = new OracleContainer("wnameless/oracle-xe-11g");
        oracleContainer.start();
        System.out.println("Oracle Container started. (" + LocalDateTime.now() + ")");
 
    }
 
    @AfterAll
    public void tearDown() {
        System.out.println("Stopping Oracle Container... (" + LocalDateTime.now() + ")");
        oracleContainer.stop();
        System.out.println("Oracle Container stopped. (" + LocalDateTime.now() + ")");
    }
 
    @Test
    public void whenSelectQueryExecuted_thenResulstsReturned() throws Exception {
 
        String jdbcUrl = oracleContainer.getJdbcUrl();
        String username = oracleContainer.getUsername();
        String password = oracleContainer.getPassword();
        Connection conn = DriverManager.getConnection(jdbcUrl, username, password);
        ResultSet resultSet = conn.createStatement().executeQuery("SELECT 1 FROM DUAL");
        resultSet.next();
        int result = resultSet.getInt(1);
 
        assertEquals(1, result);
 
    }
 
}

Maven dependencies

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.xxx.yyy</groupId>
    <artifactId>docker-testcontainers</artifactId>
    <version>0.0.1-SNAPSHOT</version>
 
    <properties>
        <java.version>1.8</java.version>
        <spring.version>5.1.3.RELEASE</spring.version>
        <testcontainers.version>1.10.2</testcontainers.version>
        <junit-engine.version>5.3.2</junit-engine.version>
        <junit-launcher.version>1.3.2</junit-launcher.version>
        <maven-compiler-plugin.version>3.7.0</maven-compiler-plugin.version>
    </properties>
 
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven-compiler-plugin.version}</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
 
    <dependencies>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>testcontainers</artifactId>
            <version>${testcontainers.version}</version>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>oracle-xe</artifactId>
            <version>${testcontainers.version}</version>
        </dependency>
        <dependency>
            <groupId>com.oracle</groupId>
            <artifactId>ojdbc6</artifactId>
            <version>12.1.0.2</version>
        </dependency>
 
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit-engine.version}</version>
            <scope>test</scope>
        </dependency>
 
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-launcher</artifactId>
            <version>${junit-launcher.version}</version>
            <scope>test</scope>
        </dependency>
         
    </dependencies>

Miscellaneous notes

Useful docker commands

• List images: docker images
• List all containers: docker ps -a
• Start a container: docker start [container id]
• List started containers: docker ps
• View logs of started container: docker logs [container id]

References

Installing Docker on Ubuntu

Further details about Post-install steps

Using an Oracle image within Docker

Database Testing With TestContainers

About Oracle 12c image

I've tried an Oracle 12c image (sath89/oracle-12c from: https://hub.docker.com/r/sath89/oracle-12c)

$ docker run -d -p 8080:8080 -p 1521:1521 --name oracle-db-12c sath89/oracle-12c

but it seems to be so slow starting up from testcontainers that the following exception is eventually (after approximately 4 minutes) thrown:

java.sql.SQLRecoverableException: ORA-01033: ORACLE initialization or shutdown in progress.

If the 12c image is started from docker host itself (i.e. Ubuntu), it does start successfully.