I want to have a CI/CD with gitlab-runner and docker swarm. I have problem when i deploy the commit will not checkout or checkout without changes, I wonder to know if problem is gitlab or docker or docker build. my .gitlab-ci.yml, look like:
stages:
- build
- deploy
build_image:
stage: build
image: docker:git
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay
script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- docker build -t registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG} ./DockerFiles/Worker
- docker push registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG}
only:
- branches
deploy_staging:
stage: deploy
image: rastasheep/ubuntu-sshd:latest
script:
# add the server as a known host
- ssh-keyscan 46.4.151.121 >> ~/.ssh/known_hosts
- chmod 600 ~/.ssh/known_hosts
# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
- eval $(ssh-agent -s)
- touch key.txt
- echo "$SSH_PRIVATE_KEY" >> key.txt
- chmod 600 key.txt
- ssh-add key.txt
# log into Docker registry
- ssh [email protected] "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com"
# stop container, remove image.
- ssh [email protected] "docker stop dockergitlab_${CI_COMMIT_REF_SLUG}" || true
- ssh [email protected] "docker rm dockergitlab_${CI_COMMIT_REF_SLUG}" || true
- ssh [email protected] "docker rmi registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG}" || true
# start new container
- ssh [email protected] "docker run --name dockergitlab_${CI_COMMIT_REF_SLUG} -d registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG}"
only:
- branches
except:
- master
and I also I put my pipeline log below, that might help to describe more:
$ eval "$CI_PRE_CLONE_SCRIPT"
00:02
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/insuretech1/backend/.git/
Created fresh repository.
From https://gitlab.com/insuretech1/backend
* [new ref] refs/pipelines/124187268 -> refs/pipelines/124187268
* [new branch] develop -> origin/develop
Checking out 735209a2 as develop...
Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
03:43
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
the content of dockerfile which I use for my build
FROM debian:buster
MAINTAINER Alireza Rahmani Khalili "[email protected]"
ENV TERM xterm
RUN apt-get update --fix-missing && apt-get install -y --force-yes curl sudo vim
RUN apt-get install -y --force-yes wget apt-transport-https lsb-release ca-certificates
RUN wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
RUN echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list
RUN echo "deb http://ftp.uk.debian.org/debian buster-backports main" >> /etc/apt/sources.list
RUN wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
RUN echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list
RUN DEBIAN_FRONTEND="noninteractive" apt-get update && apt-get install -y --force-yes \
nginx \
php7.3 \
php7.3-cli \
php7.3-fpm \
php7.3-curl \
php7.3-json \
php7.3-mysql \
php7.3-sqlite \
php7.3-xml \
php7.3-intl \
php7.3-mbstring \
php7.3-xdebug \
php-memcached \
git \
openssh-server \
php7.3-gd \
zip \
php7.3-zip
# configure php-fpm
RUN sed -i 's/^;*clear_env = .*/clear_env = no/' /etc/php/7.3/fpm/pool.d/www.conf
RUN curl -sS https://getcomposer.org/installer | php && \
mv composer.phar /usr/local/bin/composer && chmod +x /usr/local/bin/composer
RUN mkdir /var/run/sshd
RUN echo 'root:root' | chpasswd
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd
RUN echo "UseDNS no" >> /etc/ssh/sshd_config
RUN echo "KexAlgorithms diffie-hellman-group1-sha1" >> /etc/ssh/sshd_config
RUN echo "fastcgi_param PATH_TRANSLATED \$document_root\$fastcgi_script_name;" >> /etc/nginx/fastcgi_params
RUN mkdir /etc/nginx/ssl
RUN openssl ecparam -out /etc/nginx/ssl/nginx.key -name prime256v1 -genkey
RUN openssl req -new -batch -key /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/csr.pem
RUN openssl req -x509 -nodes -days 365 -key /etc/nginx/ssl/nginx.key -in /etc/nginx/ssl/csr.pem -out /etc/nginx/ssl/nginx.pem
RUN chmod 600 /etc/nginx/ssl/*
RUN ln -sf /dev/stdout /var/log/nginx/access.log
RUN ln -sf /dev/stderr /var/log/nginx/error.log
ADD docker-entrypoint.sh /usr/bin/docker-entrypoint
RUN chmod +x /usr/bin/docker-entrypoint
RUN sed -i 's/^user nginx;/user www-data;/' /etc/nginx/nginx.conf
RUN echo "apc.enable_cli=1" >> /etc/php/7.3/cli/php.ini
RUN echo "apc.shm_size=128M" >> /etc/php/7.3/fpm/conf.d/20-apcu.ini
RUN sed -i "s/\(max_execution_time *= *\).*/\1180/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(upload_max_filesize *= *\).*/\1100M/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(post_max_size *= *\).*/\1100M/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(^.*max_input_vars *= *\).*/max_input_vars = 10000/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(pm.max_children = 5\).*/\pm.max_children = 50/" /etc/php/7.3/fpm/pool.d/www.conf
RUN sed -i "s/\(pm.max_spare_servers = 3\).*/\pm.max_spare_servers = 10/" /etc/php/7.3/fpm/pool.d/www.conf
RUN echo "xdebug.default_enable=1" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN echo "xdebug.profiler_output_dir=/var/www/cachegrind/" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN echo "xdebug.profiler_enable_trigger=1" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN echo "xdebug.profiler_output_name= cachegrind.out" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN mkdir /root/.ssh/
ADD default.conf /etc/nginx/conf.d/default.conf
ADD default.conf /etc/nginx/sites-enabled/default
ADD default.conf /etc/nginx/sites-available/default
EXPOSE 22 443 80
WORKDIR /var/www/
ENTRYPOINT ["docker-entrypoint"]
CMD ["nginx", "-g", "daemon off;"]
and also content of my docker compose file which i use when I build in my ci/cd:
version: '3'
services:
worker:
image: registry.gitlab.com/insuretech1/backend:develop
ports:
- 0.0.0.0:80:80
depends_on:
- mysql
deploy:
mode: replicated
replicas: 3
# service resource management
resources:
# Hard limit - Docker does not allow to allocate more
limits:
cpus: '0.25'
memory: 512M
# Soft limit - Docker makes best effort to return to it
reservations:
cpus: '0.25'
memory: 256M
# service restart policy
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
# service update configuration
update_config:
parallelism: 1
delay: 10s
failure_action: continue
monitor: 60s
max_failure_ratio: 0.3
volumes:
- /var/www/backend:/var/www
mysql:
image: mariadb:10.4
ports:
- 0.0.0.0:3306:3306
environment:
MYSQL_ROOT_PASSWORD: root
volumes:
- /opt/mysql_data:/var/lib/mysql
deploy:
placement:
constraints: [node.role == manager]
redis:
image: redis
deploy:
placement:
constraints: [node.role == manager]
the issue is I can not see my last changes of my commit in my server (I mean i should manually git pull to fetch last changes), is there anything wrong?
first of all in your Dockerfile you should copy content of directory into docker container. that will help you keep git changes with your container, for example:
COPY . /var/www/
and other problem is in your docker compose file you have:
volumes:
- /var/www/backend:/var/www
this will override changes that git made on your container and that is why you are not able to see git changes.