I use Azure DevOps Pipelines to run terraform code. The terraform code and modules stored in connected GitHub repo to Azure Devops Pipelines. If I not use TF modules - all works fine. But If I try to use modules which stored in the same repo - I got error during Terrafrom init (when modules are downloaded):
Initializing modules...
Downloading git::https://github.com/username/terraform.git?ref=v0.0.1 for storage...
╷
│ Error: Failed to download module
│
│ Could not download module "storage" (modules.tf:1) source code from
│ "git::https://github.com/username/terraform.git?ref=v0.0.1": error
│ downloading 'https://github.com/username/terraform.git?ref=v0.0.1':
│ /usr/bin/git exited with 128: Cloning into '.terraform/modules/storage'...
│ fatal: could not read Username for 'https://github.com': terminal prompts
│ disabled
For some reason - service connection to GitHub are not enought to download modules. But checout repo, or tf code without modules works fine.
My repo looks like this:
For Terraform install, init etc I use provided plugins.
Example of part of pipeline.yml
stages:
- stage: prepare
displayName: "TF - Init and Plan"
jobs:
- job: prepare
displayName: "Prepare"
steps:
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
displayName: 'Install Terraform 1.3.6'
inputs:
terraformVersion: 1.3.7
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV3@3
displayName: 'Terraform : init'
inputs:
workingDirectory: '$(System.DefaultWorkingDirectory)/environments/dev'
backendServiceArm: '<subscription_id>'
backendAzureRmResourceGroupName: <backend_rg_name>
backendAzureRmStorageAccountName: <backend_storage>
backendAzureRmContainerName: <backend_container>
backendAzureRmKey: dev/terrafrom.state
- task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV3@3
displayName: 'Terraform : plan'
inputs:
command: plan
workingDirectory: '$(System.DefaultWorkingDirectory)/environments/dev'
environmentServiceNameAzureRM: '<subscription_id>'
backendServiceArm: '<subscription_id>'
backendAzureRmResourceGroupName: <backend_rg_name>
backendAzureRmStorageAccountName: <backend_storage>
backendAzureRmContainerName: <backend_container>
backendAzureRmKey: dev/terrafrom.state
I already tried - Solution 1 and Solution 2
But looks like it works only for code which stored in Azure Repos.
I hope exist a way how I can use Azure DevOps with code and modules in GitHub.
As this is a private GitHub repository by the error message it seems that the authorization to the GitHub repo is missing.
As per the Hashicorp Modules Sources: GitHub documentation, If using the HTTP/HTTPS protocol, or any other protocol that uses username/password credentials, configure Git Credentials Storage to select a suitable source of credentials for your environment.
Also, you can authenticate via SSH keys with GitHub. The best would be to use Deploy Keys, which is an SSH key that grants access to a single repository.
Steps:
Info: Generating a new SSH key
# Install SSH Key for private repo modules {config valid for all private Github Repos with a valid deploy key} ]
- task: InstallSSHKey@0
displayName: "Install SSH key for <repo_name> Repo"
inputs:
knownHostsEntry: $(knownHostsEntry)
sshPublicKey: $(sshPublicKey)
sshKeySecureFile: $(name_of_secure_file_in_library_group) # where private SSH key was upladed
# in Variable Group
knownHostsEntry = github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==
sshPublicKey = "SSH Public Key generated in the `Generating a new SSH key` step"
module "module_name" {
source = "github.com/username/terraform//modules/storage?ref=v0.0.4"
## CHANGE THIS TO ##
source = "[email protected]:username/terraform.git//modules/storage?ref=v0.5.1"
}
Still having doubts: Please use this step-by-step tutorial with screenshots.
https://www.codewithadam.com/using-terraform-modules-from-github-in-azure-devops/