Search code examples
terraformdatabricksazure-databricksterraform-provider-azure

How to set up unity catalog access connector with terraform

I am trying to update our Databricks account and workspace to use Unity Catalog, we've got all our infrastructure templated using terraform, I'm trying to deploy an azapi resource as detailed in the terraform unity catalog upgrade docs but am not having any luck. My terraform code below is used to create the connector

  provider "azuread" {
  client_id     = var.client_id
  client_secret = var.client_secret
  tenant_id     = var.tenant_id
}
provider "azurerm" {
  features {}
  client_id       = var.client_id
  client_secret   = var.client_secret
  subscription_id = var.subscription_id
  tenant_id       = var.tenant_id
}
provider "azapi" {
  client_id       = var.client_id
  tenant_id       = var.tenant_id
  subscription_id = var.subscription_id
  client_secret   = var.client_secret
}
resource "azurerm_resource_group" "this" {
  name     = "${local.prefix}-metaverse-migration-rg"
  location = var.region
  tags     = local.tags
}

resource "azapi_resource" "access_connector" {
type      = "Microsoft.Databricks/accessConnectors@2022-04-01-preview"
name      = "${local.prefix}-databricks-mi"
location  = azurerm_resource_group.this.location
parent_id = azurerm_resource_group.this.id
identity {
  type = "SystemAssigned"
}
body = jsonencode({
  properties = {}
})
}

the error message I receive when running this on terraform cloud is

Error: creating/updating "Resource: (ResourceId \"/subscriptions/mysubguid/resourceGroups/databricks-metaverse-migration-rg/providers/Microsoft.Databricks/accessConnectors/databricks-databricks-mi\" / Api Version \"2022-04-01-preview\")": PUT https://management.azure.com/subscriptions/mysubguid/resourceGroups/databricks-metaverse-migration-rg/providers/Microsoft.Databricks/accessConnectors/databricks-databricks-mi -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: 403 -------------------------------------------------------------------------------- { "error": { "code": "403", "message": "User not authorized." } } -----------------------------------

I'm running this under a service principal with ownership over the subscription but was getting the same error when it was a contributor. I have a suspicion I need to use a Managed Identity but not sure how/where

Thanks!

Solution

  • After adding the below permissions to the service principal, I was able to successfully deploy the access connector. For the access connector itself, I believe only the IdentityProvider.ReadWrite.All permission is needed with subsequent permissions being used for other deployment aspects

    required permissions