Search code examples
terraformdatabricksazure-databricksterraform-provider-azureterraform-provider-databricks

Terraform Databricks Storage credential for an Access Connector with User Assigned managed identity


I have a databricks access connector, that I created for accessing external locations . The access connector created has a user defined managed identity (not a system assigned one).

Now when I try to create a storage credential in Terraform using that access connector, I get an error saying

cannot create storage credential: Azure Managed Identity Credential with Access Connector Id nameofcred  could not be found

Using this:

resource "databricks_storage_credential" "storage_credential" {
  name = "example_cred"
  azure_managed_identity {
    access_connector_id = <entered the resource id for the Access Connector for Azure Databricks>
  }
  comment = "Managed identity credential managed by TF"
}

And this is how my access connector looks like (configured with userAssigned Identity)

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "accessConnectors_ac_connector_rxample": {
            "defaultValue": "example_name",
            "type": "String"
        }
    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.Databricks/accessConnectors",
            "apiVersion": "2023-05-01",
            "name": "[parameters('accessConnectors_ac_connector_example_name')]",
            "location": "northeurope",
            "identity": {
                "type": "UserAssigned",
                "userAssignedIdentities": {
                    "/subscriptions/xxxxxxxxxxxxx/resourceGroups/rg-example/providers/Microsoft.ManagedIdentity/userAssignedIdentities/userassignedminame": {}
                }
            },
            "properties": {}
        }
    ]
}

This is the terraform resource link (Azure is the cloud provider): here

Is User Assigned managed identity not supported?

I also tried with azure_service_principal block with directory_id and application_id, but it keeps failing because client_secret is a required property, and since this is a user defined managed identity, I can't create secrets (it is just listed as an SPN aka enterprise application in Azure AD).

Is this not supported?

From the GUI it is supported:

enter image description here


Solution

  • I also tried with azure_service_principal block with directory_id and application_id, but it keeps failing because client_secret is a required property, and since this is a user defined managedidentity, I can't create secrets (it is just listed as an SPN aka enterprise application in Azure AD).

    User Assigned Managed Identities are not supported in Terraform for creating storage credentials.

    Storage credential represents an authentication and authorization mechanism for accessing data stored on your cloud tenant, using either an Azure managed identity or a service principal. follow the Ms Doc about Storage credential.

    Alternatively, you can create storage credentials using a service principal by following the steps below.

    Create a service principal in the Azure portal and provide it access to your storage account.

    1. Create a client secret for the service principal and note down the directory ID , application ID and Client Secret for the service principal.
    2. Log into your storage account and assign the service principal the Storage Blob Data Contributor role. Follow the instructions in the MS doc about Data bricks storage credentials using service prinicipal

    I created a Databricks storage credential using a service principal using Terraform code

       provider "azurerm" {
      features {}
    }
    
    terraform {
      required_providers {
        azurerm = {
          source = "hashicorp/azurerm"
        }
        databricks = {
          source = "databricks/databricks"
        }
      }
    }
    
    # Use Databricks CLI authentication.
    provider "databricks" {
    profile = "DEFAULT"
    host    = "Databricks-URL"
    }
    resource "databricks_storage_credential" "storage_credential" {
      name = "databricks-storage"
      azure_service_principal {
        application_id   = ""
        directory_id     = ""
        client_secret   = ""
      }
      comment = "Service Principal credential managed by TF"
    }
    

    Terraform Plan:

    enter image description here

    Terraform Apply:

    enter image description here