Search code examples
pythonazureazure-blob-storagehttpresponseazure-data-lake-gen2

Why am I not autorized to create a file system in Datalake gen2 with Python

Using the azure.storage.filedatalake import DataLakeServiceClient in Python, I have created functionality for uploading files to my Azure datalake gen 2. This works as it should, as the files I upload indeed shows up in my Datalake, BOTH when running my code locally and with my azurewebsite.net.

Im now attempting to create functionality which can create a new container/file-system in my Datalake, which I can then upload files to. This works locally, however when I attempt this on my azurewebsite i get the following error:

HttpResponseError /project/create_file_container/
This request is not authorized to perform this operation.
ErrorCode: AuthorizationFailiure

I can see that the code fails at this specific point:

service_client = settings.SERVICE_CLIENT
# Creates a new datalake file_system for the org
system_name = 'org-1'
service_client.create_file_system(file_system=system_name) #<-- Fails on this line

My service_client is defined as:

STORAGE_ACCOUNT_NAME = 'my_datalake'
STORAGE_ACCOUNT_KEY = 'my account key'

global SERVICE_CLIENT
SERVICE_CLIENT = DataLakeServiceClient(account_url="{}://{}.dfs.core.windows.net".format(
    "https", STORAGE_ACCOUNT_NAME), credential=STORAGE_ACCOUNT_KEY)

Is there some additional authentication I have to provide when creating file_systems ? I would assume that since its possible for me to upload files, that creating file_systems would be allowed aswell.

Solution

  • I have experienced the same problem and this is a permission issue. Ensure the service principal is assigned the Blob Storage Contributor Role in the scope of the Data Lake Storage Gen2 storage account.

    Note : You can assign a role to your parent subscription or resource group, but spreading it to the storage account will require time for the assignment. If you assign the subscription or resources to the subscription or resource group and then establish the storage account, you should instantly inherit the task. When you complete a lesson, you should attach your account to the storage account directly so that it is not blocked and you do not have to wait, if you have a storage account already used.