Terraform Cloud + Github + Azure and using multiple folders with a main.tf in each of them

Question

I am trying to use github actions with Terraform Cloud on Azure Cloud. Here is my sample repo: https://github.com/fuatu/learn-terraform-github-actions-azure

The experiment was created according to the documentation from this link: https://developer.hashicorp.com/terraform/tutorials/automation/github-actions

I first tested a scenario where I have a main.tf in the root folder. It worked fine.

The next step is to have multiple folders app1 app2 and so on. So that I can change different infra items from the repo and expect the github actions to deal with it. The file is here: https://github.com/fuatu/learn-terraform-github-actions-azure/blob/main/.github/workflows/terraform.yml

As the "Terraform Plan" section was applying the code only to the root directory it was failing. I tried to change it like this below. Commented the "run" section and added a new one trying to get last commit changes and the folders and try to plan in each one of them.

      - name: Terraform Plan
        id: plan
        if: github.event_name == 'pull_request'
        #run: terraform plan -no-color -input=false
        run: |
          # Get the list of files that were changed in the latest commit
          CHANGED_FILES=$(git diff --no-commit-id --name-only -r HEAD^..HEAD)

          # Filter the list to include only the directories that contain the main.tf file
          DIRS=$(echo "$CHANGED_FILES" | grep -E '^.+/main\.tf$' | xargs -I {} dirname {} | sort | uniq)

          # Run Terraform plan in each directory that contains the main.tf file
          for DIR in $DIRS; do
            terraform plan -no-color -input=false -chdir="$DIR"
          done
        continue-on-error: true 

Even the code above seems to be working fine in my local terminal it is failing in github actions execution with the error below.

fatal: ambiguous argument 'HEAD^': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]' 

What I am trying to do is a workaround. So here is what I prefer:

1st: a better way of doing it. Maybe there are already some standards.

2nd: if the first scenario is not possible how can I make my current actions work.

Answer 1

Thanks to the comment from @GuiFalourd I managed to fix the issues in github actions.

To get the changed files I used the action below:

      - name: Get changed files
        id: changed-files
        uses: tj-actions/[email protected]

After that collected the current folder and changed folders:

      - name: Get directories
        id: directories
        run: |
          # Get the list of files that were changed in the latest commit
          CHANGED_FILES="${{ steps.changed-files.outputs.all_changed_files }}"
          echo "CHANGED_FILES"
          echo "$CHANGED_FILES"
          # Filter the list to include only the directories that contain the main.tf file
          DIRS=$(echo $CHANGED_FILES | tr -s ' ' '\n' | grep -E '^.+/main\.tf$' | xargs -I {} dirname {} | sort | uniq)
          echo "DIRS"
          echo "$DIRS"
          echo "::set-output name=dirs::$(echo $DIRS)"
          
          CURRENT_DIR=$( pwd )
          echo "Current dir: $CURRENT_DIR"
          echo "::set-output name=current::$(echo $CURRENT_DIR)"

After that for each terraform command section looped. Example below:

      - name: Terraform Validate
        id: validate
        run: |
          for DIR in ${{ steps.directories.outputs.dirs }}; do
            echo "Dir to validate: $DIR"
            cd "${{ steps.directories.outputs.current }}"
            cd "$DIR"
            terraform validate -no-color
          done

full details here: https://github.com/fuatu/learn-terraform-github-actions-azure/blob/main/.github/workflows/terraform.yml

As a last comment. I think this is a suboptimal solution and terraform should provide something better.

If someone can provide any best practices, it will be great.