Search code examples
dockergitlabgitlab-ciamazon-ecs

Facing problems passing env files to docker container using Gitlab Ci/Cd for Amazon ECS

I have scanned every possible article on the internet on this topic since yesterday, but I am just not able to figure out how to pass my env files onto the container running on Gitlab.

I am working on NodeJs app using Gitlab Ci/CD, DockerFile, and Amazon ECS for deploying my project onto the service. I have 3 different env files for 3 different environments i.e., prod, staging and the dev.

here is my .gitlab-ci.yml:

include:
  - template: AWS/Deploy-ECS.gitlab-ci.yml

Dockerfile contents are as below:

FROM node:16 

WORKDIR /.

# Copy package.json and package-lock.json   
COPY ./package*.json ./

# Install dependencies
RUN npm install

# 
COPY . .

# Build the application
RUN npm run build

# Expose port 3000
EXPOSE 3000

# Start the server

CMD [ "npm", "run", "prod" ]

the .env files are in the local bin folder and each file has many env variables inside:

enter image description here

I cannot seem to pass these items while pipeline builds, and it results in failure. Not only that but I am not even able to create my ECS service because app is not able to bootstrap correctly on the cluster because of the missing env files.

What I tried:

Almost everything. To summarize here are the steps I followed:

  • I Read about docker compose on how to import env files based on dev, prod or stag environment but in my case, I don't know how to use docker compose for ECS as I have manually done my configuration on Amazon ECS by creating Task definitions, Clusters and even tried running demo tasks by myself. Now all I have to do is configure ci/cd via gitlab.

  • Added a base docker-compose to my app:

enter image description here

but not able to modify .gitlab-ci.yml to run docker-compose in some way, as I am simply including Deploy-ECS.gitlab-ci template as shown above. If I write my own gitlab-ci then it will not auto build my changes once I push on the branch.

  • I read about Deploying on ECS cluster using Gitlab Ci/Cd on their official webpage but it is very generic use case and explains nothing on env files.

Since I am stuck on this since yesterday, it is getting very exhausting for me to fully understand how to make this work. Please guide me.

Solution

  • Judging from your setup is hard to exactly tell how you are using GitLab - but everything seems to point that you are making use of AutoDevops so I'll try to help.

    The template you are importing, it's using this other CI file which is in charge of building the Docker image. In this template, the build.sh is executed and that file is hosted in another GitLab project, visible here.

    Given that everything is provided by GitLab and it's a bit opaque for the end user, you have to dig into the AutoDevops config variables to know how to pass custom variables to the build job.

    Now coming into the Docker specifics, you can use the arg -env-file but only with Docker Compose so that won't work here. If you have only a few variables to add, you can add them in the AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS variable to have them forwarded to the Docker container:

    include:
  - template: AWS/Deploy-ECS.gitlab-ci.yml

variables:
  AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS: --build-arg=YOUR_VARIABLE=foobar

    That setup is not ideal if you have lots of env vars so you can also add the .env file on your Docker image:

    ...

COPY env.dev .
RUN source env.dev

...

    Hope this helps or guides you in the correct direction!