Search code examples
azureazure-pipelinesazure-storagepipelineazure-synapse-analytics

Call Azure Data Lake Storage Gen2 REST APIs using the web activity in an Azure Synapse pipeline

How can I use the Azure Data Lake Storage Gen2 REST API to rename a folder in an Azure Storage Account using the web activity in a Synapse pipeline? The folders are in the same container, and I need to rename them by executing a pipeline.

Solution

  • To rename a folder in ADLS Gen 2 using REST API and web activity, follow the procedure below:

    First, add the storage blob data contributor role to your Synapse workspace managed identity as follows:

    Step 1: Go to the IAM of the ADLS account and click on "Add role assignment" as shown below:

    enter image description here

    Step 2: Search for the storage Blob data contributor role and select it, as shown below:

    enter image description here

    Step 3: Select the Managed identity, as shown below:

    enter image description here

    After successfully assigning the role, go to the Synapse workspace, create a pipeline with web activity, and configure the web activity as follows:

    Mention the URL in the format below:

     https://<storageAccountName>.dfs.core.windows.net/<containerName>/<newNameOfFolder>

    Select the PUT method and add @toLower('') dynamic content to the body. Select System Assigned Managed Identity as the authentication method with https://storage.azure.com resource. Add x-ms-rename-source as a header with the value /<containerName>/<oldNameOfFolder>, as shown below:

    enter image description here

    Debug the pipeline; it will run successfully and rename the folder. You can check the images below:

    Before renaming:

    enter image description here

    After renaming:

    enter image description here