Search code examples
databricksazure-databricks

Using service principal to access blob storage from Databricks

I followed Access an Azure Data Lake Storage Gen2 account directly with OAuth 2.0 using the Service Principal and want to achieve the same but with blob storage general purpose v2 (with hierarchical fs disabled). Is it possible to get this working, or authenticating using access key or SAS is the only way?

Solution

  • No that is not possible as of now. OAuth Bearer Token is supported for Azure Data Lake Storage Gen2 (with the hierarchical namespace enabled when creating the storage account). To access Azure Data Lake Store Gen2 the ABFS-driver is used:

    abfss://<your-file-system-name>@<your-storage-account-name>.dfs.core.windows.net/

    To access the Blob Storage you use WASB:

    wasbs://<your-container-name>@<your-storage-account-name>.blob.core.windows.net

    only supporting token based access.