In my GitLab CI build, I want to:
8000 in docker, port 8000 exposed. Of course, everything works locally, I can connect (curl, awc-cli, Java code for Amazon DB, whatever you wish).--endpoint-url http://localhost:8000 (or any other mapped IP instead of localhost)..gitlab-ci.yml looks like this:
image: docker:stable
build/test:
tags:
- gradle
- eu
stage: test
# doesn't work with or without it
# services:
# - docker:dind
script:
# display local running containers
- echo Displaying all running docker containers with "amazon/dynamodb-local" image...
- docker ps --filter ancestor=amazon/dynamodb-local
# stop all running docker containers with "amazon/dynamodb-local" image
- echo Stopping all Docker containers with "amazon/dynamodb-local" image...
- CONTAINERS=$(docker ps -q --filter ancestor=amazon/dynamodb-local)
- >
if [ "${CONTAINERS}" == "" ]; then
echo No docker containers with "amazon/dynamodb-local" image running. Nothing to stop.
else
docker stop $(docker ps -q --filter ancestor=amazon/dynamodb-local)
echo All Docker containers with "amazon/dynamodb-local" image stopped.
fi
# start DynamoDB local as a docker container with shared database
# - java -Djava.library.path=./dynamodb_local_latest/DynamoDBLocal_lib -jar ./dynamodb_local_latest/DynamoDBLocal.jar -sharedDb
# relative path to causes "Error: Unable to access jarfile" for both windows and linux
# run Docker in detached mode to not hang on the opened console
- cd ./dynamodb_local_latest
- docker run --detach -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -sharedDb
- cd ./..
# display local running containers
- echo Displaying all running docker containers with "amazon/dynamodb-local" image...
- docker ps --filter ancestor=amazon/dynamodb-local
# see https://stackoverflow.com/questions/45389116/unable-to-access-docker-compose-containers-created-inside-docker
# $DOCKER_HOST is unix:///var/run/docker.sock
# http://localhost:8080 fails
# http://docker:8000 fails
# http://unix:///var/run/docker.sock:8000 fails
- echo docker host is ${DOCKER_HOST}
- cat /etc/hosts
# - curl docker:80 | true
# - curl docker:8000 | true
# - curl http://docker:8000 | true
# - curl http://docker:8000 | true
# - curl ${DOCKER_HOST} | true
# - curl ${DOCKER_HOST}:8000 | true
- curl localhost:8000 | true
- curl http://localhost:8000 | true
# stop all running docker containers with "amazon/dynamodb-local" image
- echo Stopping all Docker containers with "amazon/dynamodb-local" image...
- CONTAINERS=$(docker ps -q --filter ancestor=amazon/dynamodb-local)
- >
if [ "${CONTAINERS}" == "" ]; then
echo No docker containers with "amazon/dynamodb-local" image running. Nothing to stop.
else
docker stop $(docker ps -q --filter ancestor=amazon/dynamodb-local)
echo All Docker containers with "amazon/dynamodb-local" image stopped.
fi
# display local running containers
- echo Displaying all running docker containers with "amazon/dynamodb-local" image...
- docker ps --filter ancestor=amazon/dynamodb-local
Critical execution points (Gitlab-CI log) look like this:
Docker container runs:
$ docker run --detach -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -sharedDb
c823489c22fffa603c1ae1b91d898cb7de4964774d54a08c9fdf0b891c2243b4
$ echo Displaying all running docker containers with "amazon/dynamodb-local" image...
Displaying all running docker containers with amazon/dynamodb-local image...
$ docker ps --filter ancestor=amazon/dynamodb-local
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c823489c22ff amazon/dynamodb-local "java -jar DynamoDBL…" 1 second ago Up Less than a second 0.0.0.0:8000->8000/tcp peaceful_beaver
curl fails (tried all possible variations, this is just an example):
$ curl localhost:8000 | true
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (7) Failed to connect to localhost port 8000: Connection refused
Authenticating with credentials from $DOCKER_AUTH_CONFIG
ERROR: Job failed: exit code 7
I tried with and without
services:
- docker:dind
Tried with host names localhost or docker or tcp://localhost, with http:// prefix or without it, with ports 80, 8000 or 2375. Nothing works.
Value of ${DOCKER_HOST} is unix:///var/run/docker.sock:
$ echo docker host is ${DOCKER_HOST}
docker host is unix:///var/run/docker.sock
/etc/hosts does not contain an alias for docker
$ cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.3 runner-H3HL9s9t-project-913-concurrent-0
docker run, or complex docker-compose usage required.docker alias in /etc/hosts?unix:///var/run/docker.sock a valid value of ${DOCKER_HOST}? Shouldn't the value for this variable be set in variables of .gitlab-ci.yml (particularly to tcp://localhost:2375)?There are multiple links that I've googled. Solutions from there haven't helped me for now.
docker-composeThere is actually no need in writing such a complex solution with manual docker run / docker stop inside the -script section. The slim and simple way is to use the local DynamoDB as service.
With this script, local DynamoDB will be accessible via url from alias of the services element, i.e. dynamodb-local for this example:
services:
- name: amazon/dynamodb-local
alias: dynamodb-local
Executing aws dynamodb with http://dynamodb-local:8000 endpoint url works:
script:
- DYNAMODB_LOCAL_URL=http://dynamodb-local:8000
- apk add --no-cache curl jq python py-pip
- pip install awscli
- aws dynamodb list-tables --endpoint-url ${DYNAMODB_LOCAL_URL} --region eu-central-1
This option was found in this great answer. Many thanks to @madhead for providing it!