How to connect Azure Databricks and ADLS Gen 2 using Managed Identity?

Question

How to connect Azure Databricks and ADLS Gen 2 using Managed Identity?

As per the Microsoft documentation link below, I used unity catalog and enabled managed identity access to connect ADLS Gen 2 and Databricks but it works only for the managed tables creation (adls location of the managed tables is happening using managed identity), not able to manage files using notebooks, is there any option available to manage adls files(read/write) from azure databricks using managed identity?

https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/azure-managed-identities#use-a-managed-identity-to-access-storage-managed-by-a-unity-catalog-metastore

Answer 1

Thanks @Alex Ott making as an answer it will useful to other community members.

How to connect Databricks and ADLS Gen 2 using Managed Identity?

Using external location and storage credentials it allows unity catalog to read and write data. allows to connect Azure Databricks and ADLS Gen 2 using Managed Identity.

• Create external table for read and write operation in ADLS gen2.

Note: External location that combines a cloud storage path with a storage credential that authorizes access to the storage path. the request fails, If a user does not have access to a storage location in unity catalog.

For more information refer this MS document.