Github actions build docker image when there are changes in folder

Question

Net project. I have multiple projects running and all projects are dockerized. currently I am using matrix statergy to build all the docker files. Here I want to change my implementation. I have many projects/services so whenever there is change I do not want to build and deploy all the projects/services but i want to deploy specific project/service. So I am looking for the folder change approach. So whenever there is change in folder I am planning to build only that projet/service.

Below is my current approach using matrix

build-and-push-image:
    name: "BUILD: ${{ matrix.name }}"
    runs-on: ubuntu-latest
    needs: prep
    strategy:
      matrix: ${{fromJson(needs.prep.outputs.matrix)}}
    permissions:
      contents: read
      packages: write

    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
      
      - name: Log in to the Container registry
        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
        with:
          images: ${{ matrix.docker-image-name }}
          flavor: latest=true
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
      - name: Restore dependencies
        shell: bash
        run: dotnet restore
        
      - name: Build and push Docker image ${{ matrix.name }}
        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
        with:
          context: ${{matrix.context }}
          file: ${{ matrix.dockerfile }}
          push: true
          tags: ${{ matrix.docker-image-name }}:${{ github.sha }}

Using this approach it builds all the dockerfiles but i want to build and deploy specific project/service when there is change. So how Can I do this? Or is there any good approach to handle this? Can someone please help me? Thanks in advance

Answer 1

Let's say that we have a repository like:

.
├── .github
│   └── workflows
│       └── matrix_depend_on_changes.yml
├── .gitignore
├── proj1
│   └── test1.txt
├── proj2
│   └── test2.txt
└── proj3
    └── test3.txt

If any file of proj1 directory is changed we have to build docker-image-1 image. Same with proj2 and proj3. If no changes in proj1 then docker-image-1 build should be skipped.

In this case matrix_depend_on_changes.yml may be looks like:

---
name: 'triggered on PUSH'

on:
  push:
    branches:
      - your_brunch_name

jobs:
  get_with_predefined_action:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
      pull-requests: read

    steps:
      # Way is recommended by the Internet
      - name: Checkout Code
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Get changed files with predefined action
        id: changed-files
        uses: tj-actions/changed-files@v37
  
      - name: List all changed files
        run: |
          echo '# FILES with tj-actions/changed-files' >> "${GITHUB_STEP_SUMMARY}"
          echo '' >> "${GITHUB_STEP_SUMMARY}"
          echo '```' >> "${GITHUB_STEP_SUMMARY}"

          for file in ${{ steps.changed-files.outputs.all_changed_files }}; do
            echo "$file was changed" >> "${GITHUB_STEP_SUMMARY}"
          done

          echo '```' >> "${GITHUB_STEP_SUMMARY}"

  get_manually:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
      pull-requests: read
    outputs:
      IMAGES_TO_BUILD: ${{ steps.set_images.outputs.IMAGES_TO_BUILD }}
    steps:
      # Dirty way (Personally I like it)
      - name: Checkout Code
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Get changed files manually
        run: |
          files_list="$(git diff --name-only ${{ github.event.before }} HEAD | xargs)"
          echo '# Files with git command' >> "${GITHUB_STEP_SUMMARY}"
          echo '' >> "${GITHUB_STEP_SUMMARY}"
          echo '```' >> "${GITHUB_STEP_SUMMARY}"

          for file in ${files_list}; do
            echo "$file was changed" >> "${GITHUB_STEP_SUMMARY}"
          done

          echo '```' >> "${GITHUB_STEP_SUMMARY}"

          # save files into variable
          printf 'THE_FILES=%s\n' "${files_list}" >> "${GITHUB_ENV}"

      # Since output of tj-actions/changed-files and 
      - name: Get docker image names
        # I'm familiar with python so will use it to parse file names
        # be careful if you have spaces into file names
        shell: python
        env:
          PROJECT_IMAGE_MAP: '{"proj1": "docker-image-1", "proj2": "docker-image-2", "proj3": "docker-image-3"}'
        run: |
          from os import environ
          import json
          
          # split files string into list. Divide by spaces
          files = environ.get("THE_FILES").split(' ')
          proj_image_map = json.loads(environ.get("PROJECT_IMAGE_MAP"))

          images = []

          # check if proj1, proj2, etc is a part of changed file path
          for file in files:
            for k, v in proj_image_map.items():
              if k in file:
                if v not in images:
                  images.append(v)

          # save retreived images as json array
          with open(environ.get("GITHUB_ENV"), 'a') as f:
            f.write('IMAGES_TO_BUILD=' + json.dumps(images) + '\n')
      
      - name: Set image names as output for matrix
        id: set_images
        run: |
          echo "IMAGES_TO_BUILD=${IMAGES_TO_BUILD}" >> $GITHUB_OUTPUT

  print_image_names_with_matrix:
    name: "Build image: ${{ matrix.image }}"
    runs-on: ubuntu-latest
    needs: get_manually
    strategy:
      matrix:
        image: ${{ fromJson( needs.get_manually.outputs.IMAGES_TO_BUILD ) }}

    steps:
      - name: Print image name from matrix
        run: |
          echo "${{ matrix.image }}"

Here is get_with_predefined_action and get_manually jobs are doing the same stuff. Checking for changed files. However, I don't like to use third party actions that may be written in couple of lines manually, so I chose second one to get files using git command. Steps Get docker image names and Set image names as output for matrix may be used with both types of jobs above.

I'm using PROJECT_IMAGE_MAP environment variable to associate folder names with image names. And IMAGES_TO_BUILD output will be used to build exact images (Of course it is a sample and Docker image names are just printed out)

Finally if I change proj1/test1.txt and proj3/test3.txt files and push new commit, Summary will be looks like:

---

FILES with tj-actions/changed-files

proj1/test1.txt was changed
proj3/test3.txt was changed

Files with git command

proj1/test1.txt was changed
proj3/test3.txt was changed

And workflow is looking like:

I hope that is expected behavior.

Warning:

Be careful with files with spaces. Something like /home/User Folder will be splitted into two paths /home/User and Folder. You should consider to fix logic of Get changed files manually and Get docker image names steps if you plan to use files with spaces.