Search code examples
azureazure-data-factoryazure-synapse

How to export Adf pipelines,dataflows,datasets into Azure Synapse

I have 100 dataflows ,50 pipelines and their related datasets variables etc. Now i want to use Synapse service and want my all pipelines amd stuff of ADF into Synapse. My Adf is git configured Can we export them in one go??

Solution

  • To move data from data factory to synapse we need to download the supported module files in ADF.

    enter image description here

    I tried to migrate data from ADF to Synapse workspace using PowerShell following below procedure:

    I connected to Azure account using below command:

    Connect-AzAccount

    I set my subscription using below command:

    Select-AzSubscription -SubscriptionName "<SubscriptionName>"

    enter image description here

    I created a linked service for source and sink using below comand:

    Set-AzSynapseLinkedService -WorkspaceName <synapseworkspace> -Name <linkedservicename> -DefinitionFile "<json file path>"

    I created dataset for source and sink in synapse workspace using below command:

    Set-AzSynapseDataset  -WorkspaceName <synapseworkspace> -Name <datasetname> -DefinitionFile "<json file path>"

    I created pipeline in synapse workspace using below command:

    Set-AzSynapsePipeline -WorkspaceName <synapseworkspace> -Name <pipelinename> -DefinitionFile "<json file path>"

    They are created successfully in synapse workspace:

    enter image description here

    In your case you are having bulk data in your data factory, you can use Azure Data Factory to Synapse Analytics Migration Tool.

    You can migrate Azure Data Factory pipelines, datasets, linked service, integration runtime and triggers to a Synapse Workspace through below PowerShell script:

      .\importADFtoSynapseTool.ps1 -sourceADFResourceId "/subscriptions/<SubscriptionID>/resourcegroups/<ADFResourceGroupName>/providers/Microsoft.DataFactory/factories/<ADFName>" -destSynapseResourceId "/subscriptions/<SubscriptionID>/resourcegroups/<SynapseResourceGroupName>/providers/Microsoft.Synapse/workspaces/<SynapseName>" -TenantId <tenantId>
    • Resource group name for Azure Data Factory
    • Resource group name for Azure Synapse Workspace
    • Subscription ID
    • Tenant ID (you can find this by click Azure Active Directory in the Azure Portal)

    For more clarification you can visit this.