Search code examples
gitsshazure-devopsterraformazure-repos

How to source Terraform module hosted within Azure DevOps Repo via ssh

Looking for the correct/working way to source a Terraform module that is hosted within a private Azure DevOps git repo via SSH (see TF Docs on Git repo sourcing via ssh).

It's undesirable to have a PAT token as it will eventually expire and renewal can't be automated yet (although coming soon apparently). Despite that, have validated the https method but I would like to avoid needing to edit the source if I can as module source references can't include terraform vars at init time.

I have followed the following steps to generate an ssh private/public key pair and those files have been added to my local .ssh folder @ C:\Windows\Users\<username>\.ssh.

Example terraform code below:

module "test" {
    source = "<ref>"
}

Where <ref> has been tested with:

  • git::https://<pat token>:dev.azure.com/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref> - worked/validated https approach as last resort
  • git::ssh://git@ssh.dev.azure.com:v3/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref>
  • git::ssh://git@ssh.dev.azure.com/v3/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref> - see terraform/issues/18869

After reading about multiple keys here:

Generally, if you configure multiple keys for an SSH client and connect to an SSH server, the client can try the keys one at a time until the server accepts one. However, this doesn't work with Azure DevOps for technical reasons related to the SSH protocol and how our Git SSH URLs are structured. Azure DevOps will blindly accept the first key that the client provides during authentication. If that key is invalid for the requested repo, the request will fail with the following error: remote: Public key authentication failed. fatal: Could not read from remote repository.

I played around with my .ssh/config file including

  • Removing all other keys other than the automation ssh key pair that I wanted to use specifically for this case (as opposed to my own personal SSH key identifying me)
  • Updating .ssh/config to look like:
Host automation
    HostName ssh.dev.azure.com
    IdentityFile ~/.ssh/automation_account
    IdentitiesOnly yes
Host azuredevops
    HostName ssh.dev.azure.com
    IdentityFile ~/.ssh/azuredevops
    IdentitiesOnly yes

And then within the <ref> trying:

  • git::ssh://git@automation/v3/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref>
  • git::ssh://automation/v3/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref>

Where all of these attempts result in:

Permission denied, please try again.
git@ssh.dev.azure.com: Permission denied (password,publickey).
fatal: Could not read from remote repository.

Or

does not appear to be a git repository
fatal: Could not read from remote repository.

The ssh key that I am using, have validated that it is registered with Azure DevOps by getting the thumbprint ssh-keygen -E md5 -lf <path to private key> and getting back 2048 MD5:ab:e2:... automation (RSA) which I can confirm exists within Azure DevOps under my SSH keys.

I have validated that the key allows me to access one of my target TF repos by adding a new git remote using that specific key. git remote -v gives me:

aorigin       git@automation:v3/<org name>/<project name>/_git/<repo name> (fetch)
aorigin       git@automation:v3/<org name>/<project name>/_git/<repo name> (push)
origin  git@ssh.dev.azure.com:v3/<org name>/<project name>/_git/<repo name> (fetch)
origin  git@ssh.dev.azure.com:v3/<org name>/<project name>/_git/<repo name> (push)

And a git pull aorigin works as expected. Origin is the original SSH clone url from the Azure DevOps gui.

Almost certain that I am missing something obvious but after extensive Googling and a number of different configurations, cannot for the life of me get it to work. Help/pointers/suggestions appreciated

Versions of tools:

  • Terraform v0.13.4
  • git version 2.17.1.windows.2
  • OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
Solution

  • Following this issue and the Terraform documentation, I would try the URL

    git::automation:v3/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref>

    (Adding User git to my .ssh/config file, right after Hostname, in order to not have to add the user)

    The OP Jamie performed the following steps:

    • Updated Git to mentioned version
    • Regenerated some new keys using steps in MS Docs
    • Added all keys using ssh-add
    • Added public keys to Azure DevOps
    • Made the following changes to .ssh/config file:
    Host ssh.dev.azure.com 
  IdentityFile ~/.ssh/me
  IdentitiesOnly yes

Host automation
  HostName ssh.dev.azure.com
  User git
  IdentityFile ~/.ssh/auto 
  IdentitiesOnly yes 

Host automationsec 
  HostName ssh.dev.azure.com 
  User git 
  IdentityFile ~/.ssh/auto_sec 
  IdentitiesOnly yes

    Tested with the two following cases:

    module "test" {
  source     = "git::automation:v3/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref>"
}

    and

    module "test" {
  source     = "git::automationsec:v3/<org name>/<project name>/_git/<repo name>//<sub path>?ref=<version ref>"
}

    The first passwordless entry works, the second case doesn't. Unsure of the root cause as to why a ssh key with a passphrase isn't accepted