Databricks Unity Catalog - Error: cannot create metastore data access

Question

I'm in the progress of enabling Databricks Unity Catalog and encountered a problem with the databricks_metastore_data_access Terraform resource:

resource "databricks_metastore_data_access" "this" {
  provider = databricks.account-level

  metastore_id  = databricks_metastore.this.id
  name          = "mi-access-databricks-metastore"
  owner         = "admin-group"
  force_destroy = true
  is_default    = true

  azure_managed_identity {
    access_connector_id = azurerm_databricks_access_connector.this.id
  }
}

The Terraform apply run will always fail with following error: Error: cannot create metastore data access
Unfortunately I get no more error details than the message from above.

The funny thing is that all other Unity Catalog related resource were created successfully and looking at the Account Console I can see that the metastore was assigned to my Databricks workspace.

So my question: Do you know for what the metastore data access resource is used for and do I even need it? If yes, how can I fix this issue?

Answer 1

I did not find a direct solution to this problem, however we don't need this resource anymore at all. Turns out you can also create a Metastore without the underlying storage infrastructure. As we have environment separated external storage locations anyway, the Metastore storage account is obsolete for us. This means the only two resources we need in order to configure the Metastore is to just use the databricks_metastore and the databricks_metastore_assignment resource. This important fact is still missing in the official documents and should be updated.