Search code examples
azureazure-data-factoryazure-synapse-pipeline

If using Azure Synapse Analytics in a new project - is it best to use Synapse Pipelines over ADF, or are there major considerations?

We're moving from on-premise to Azure and will be using Azure Synapse. Our transformations will largely be lift-and-shift of SQL stored procs to start.

Looking at the high-level documentation, I'm not seeing any differences between ADF & Azure Synapse Pipelines that would matter for our use case: MS Overview

Given we'll be using Azure Synapse Analytics, does it make the most sense to use Synapse Pipelines over ADF, just to be a bit more streamlined, or are there other considerations?

i.e. could pricing be totally different, documentation/support (I assume ADF would be better...but maybe they're so similar it's a moot point), etc.

Solution

  • Everything depends on how you want to design and architect because there are still couple of discrepancies between Synapse pipelines and ADF:

    1. IRs cannot be shared between Synapse workspaces unlike ADFs
    2. AIRFlow integration and Power Query are in ADf and still not in synapse

    https://azurede.com/2021/03/27/difference-between-synapse-pipelines-and-azure-data-factory-pipelines/