Search code examples
apache-sparkazure-databricksazure-data-lake-gen2

I am unable to mount ADLS Gen2. Please assist

Code to mount ADLS Gen2:

enter image description here

Error while mounting ADLS Gen2:

enter image description here

Solution

  • From the error message it clear says - the provided tenant is not found in your subscription. Please make sure the tenant ID is available in your subscription.

    Prerequisites:

    Create and grant permissions to service principal If your selected access method requires a service principal with adequate permissions, and you do not have one, follow these steps:

    Step1: Create an Azure AD application and service principal that can access resources. Note the following properties:

    • application-id: An ID that uniquely identifies the application.
    • directory-id: An ID that uniquely identifies the Azure AD instance.
    • service-credential: A string that the application uses to prove its identity.
    • storage-account-name: The name of the storage account.
    • filesystem-name: The name of the filesystem.

    Step2: Register the service principal, granting the correct role assignment, such as Storage Blob Data Contributor role, on the Azure Data Lake Storage Gen2 account.

    Step3: Azure Data Lake Gen2 mount by passing the values directly.

    configs = {"fs.azure.account.auth.type": "OAuth",
       "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
       "fs.azure.account.oauth2.client.id": "06ecxx-xxxxx-xxxx-xxx-xxxef", #Enter <appId> = Application ID
       "fs.azure.account.oauth2.client.secret": "1i_7Q-XXXXXXXXXXXXXXXXXXgyC.Szg", #Enter <password> = Client Secret created in AAD
       "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/72f98-xxxx-xxxx-xxx-xx47/oauth2/token", #Enter <tenant> = Tenant ID
       "fs.azure.createRemoteFileSystemDuringInitialization": "true"}

dbutils.fs.mount(
source = "abfss://<filesystem>@<StorageName>.dfs.core.windows.net/<Directory>", #Enter <container-name> = filesystem name <storage-account-name> = storage name
mount_point = "/mnt/<mountname>",
extra_configs = configs)

    enter image description here

    For more details, refer to Azure Databricks - Azure Data Lake Storage Gen2.