running shell script with terraform provisioner "local-exec" returns permissions denied in Azure DevOps returns permissions denied

Question

I'm trying to provision a databricks with a pat token with a null_resource and local-exec. this is the code block:

resource "null_resource" "databricks_token" {
  triggers = {
    workspace = azurerm_databricks_workspace.databricks.id
    key_vault_access = azurerm_key_vault_access_policy.terraform.id
  }
  provisioner "local-exec" {
    command = "${path.cwd}/generate-pat-token.sh"
    environment = {
      RESOURCE_GROUP = var.resource_group_name
      DATABRICKS_WORKSPACE_RESOURCE_ID = azurerm_databricks_workspace.databricks.id
      KEY_VAULT = azurerm_key_vault.databricks_token.name
      SECRET_NAME = "DATABRICKS-TOKEN"
      DATABRICKS_ENDPOINT = "https://westeurope.azuredatabricks.net"
    }
  }
}

however, I get the following error:

2020-02-26T19:41:51.9455473Z [0m[1mnull_resource.databricks_token: Provisioning with 'local-exec'...[0m[0m
2020-02-26T19:41:51.9458257Z [0m[0mnull_resource.databricks_token (local-exec): Executing: ["/bin/sh" "-c" "/home/vsts/work/r1/a/_Infrastructure/Infrastructure/ei-project/devtest/generate-pat-token.sh"]
2020-02-26T19:41:51.9480441Z [0m[0mnull_resource.databricks_token (local-exec): /bin/sh: 1: /home/vsts/work/r1/a/_Infrastructure/Infrastructure/ei-project/devtest/generate-pat-token.sh: Permission denied
2020-02-26T19:41:51.9481502Z [0m[0m
2020-02-26T19:41:52.0386092Z [31m
2020-02-26T19:41:52.0399075Z [1m[31mError: [0m[0m[1mError running command '/home/vsts/work/r1/a/_Infrastructure/Infrastructure/ei-project/devtest/generate-pat-token.sh': exit status 126. Output: /bin/sh: 1: /home/vsts/work/r1/a/_Infrastructure/Infrastructure/ei-project/devtest/generate-pat-token.sh: Permission denied
2020-02-26T19:41:52.0401076Z [0m
2020-02-26T19:41:52.0401373Z 
2020-02-26T19:41:52.0401978Z [0m[0m[0m

side note, this is with Azure DevOps

Any idea how to solve the permission denied ?

Answer 1

The root of this problem is with how Azure DevOps stores artifacts and repositories. Here is a snippet from their documentation explaining why this happens.

https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/build-artifacts?view=azure-devops&tabs=yaml#download-to-debug

Under the TIPS, section you will see the following:

• Build artifacts are stored on a Windows filesystem, which causes all UNIX permissions to be lost, including the execution bit. You might need to restore the correct UNIX permissions after downloading your artifacts from Azure Pipelines or TFS.

This means that your files downloaded (in this case your shell script) have all unix permissions wiped. To fix this problem, I add a step to first set the proper permissions on the shell script before executing the shell script. See the below example where I have added the fix to the code you provided.

resource "null_resource" "databricks_token" {
  triggers = {
    workspace = azurerm_databricks_workspace.databricks.id
    key_vault_access = azurerm_key_vault_access_policy.terraform.id
  }
  provisioner "local-exec" {
    command = "chmod +x ${path.cwd}/generate-pat-token.sh; ${path.cwd}/generate-pat-token.sh"
    environment = {
      RESOURCE_GROUP = var.resource_group_name
      DATABRICKS_WORKSPACE_RESOURCE_ID = azurerm_databricks_workspace.databricks.id
      KEY_VAULT = azurerm_key_vault.databricks_token.name
      SECRET_NAME = "DATABRICKS-TOKEN"
      DATABRICKS_ENDPOINT = "https://westeurope.azuredatabricks.net"
    }
  }
}

The command section will first set the execute permissions on the shell script and then execute it.