Github Actions with Private Github Repository as Source for Terraform Modules

Question

I have a github organization account under which there are two repositories.

• tf-modules ( Reusable codes are written here )
• tf-consumer ( Modules are consumed here )

I'm trying to use the "github.com/custom-organization/tf-modules/modules/ec2" as source in the main.tf files of tf-consumer

I'm not owner for both the repositories, but I have admin privileges only for tf-consumer github repository to create any secrets just in case.

tf-consumer directory structure is as follows:

• tf-consumer/stacks/prod/main.tf
• tf-consumer/stacks/dev/main.tf

Example main.tf is as follows:

module "ec2_creator" {
    source = "git::https://github.com/custom-organization/tf-modules.git//modules/ec2"
    // following lines of code here
    // following lines of code here
}

Github actions workflow is as follows:

name: Terraform

on:
  pull_request:
    branches:
      - mgmt-scp
    types: [opened, synchronize]

  push:
    branches:
      - mgmt-scp

env:
  AWS_DEFAULT_REGION: "us-east-1"

jobs:
  terraform:
    name: Terraform
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: "ACCESS-KEY"
          aws-secret-access-key: "SECRET-KEY"
          aws-region: ${{ env.AWS_DEFAULT_REGION }}

      - name: Install Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          terraform_version: ${{ env.TERRAFORM_VERSION }}

      - name: Terraform Init
        working-directory: ./stacks/sampler-scp
        command: |
          git config --global url."https://my-username:${MY_GH_TOKEN}@github.com/custom-organization".insteadOf "https://github.com/custom-organization"
          terraform init -input=false
        env:
          MY_GH_TOKEN: ${{ secrets.CUSTOM_TOKEN }}

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        if: github.event_name == 'pull_request'
        run: terraform plan -input=false -out=tfplan

      - name: Terraform Apply
        if: github.event_name == 'push'
        run: terraform apply -input=false tfplan

I keep getting following error

│ Error: Failed to download module
│ 
│ Could not download module "ec2_creator" (main.tf:1) source code from
│ "git::https://github.com/custom-organization/tf-modules.git": error downloading
│ 'https://github.com/custom-organization/tf-modules.git': /usr/bin/git exited with
│ 128: Cloning into '.terraform/modules/ec2_creator'...
│ remote: Invalid username or password.
│ fatal: Authentication failed for
│ 'https://github.com/custom-organization/tf-modules.git/'

Things i've tried

• I've created a personal access token from Developer settings > Personal Access tokens > Tokens ( Classic ) with following permissions
  • Full Control of private repositories
  • workflow
  • read org and team memberships, read org projects
  • manage org runners and runner groups
  • read user public keys
  • read repository hooks
  • read all user profile data
  • access user email addresses ( read-only )
  • read access of projects
  • read public user gpg keys
  • read public user ssh signing keys
• I've created a secret under mgmt-scp environment a secret called as "CUSTOM_TOKEN" and provided my PAT value in it and referenced in above code.

Expectation is when terraform does initialization, it should read my tf-modules repository mentioned as source

What else am I missing here ? Is there a medium article of steps performed above to resolve issue should also be helpful

Answer 1

Try with:

  - name: Terraform Init
    working-directory: ./stacks/sampler-scp
    command: |
      echo "machine github.com login x password ${MY_GH_TOKEN}" > ~/.netrc
      git config --global url."https://github.com/".insteadOf "git://github.com/"
      git config --global advice.detachedHead false
      terraform init -input=false
    env:
      MY_GH_TOKEN: ${{ secrets.CUSTOM_TOKEN }}