Search code examples
azure-synapsedataverseazure-synapse-analytics

Azure Synapse Link for Dataverse - Tables from F&O - only available via Spark pool for Delta Lake?


I am migrating a project from one tenant to another, which in essence is ETL job from Dynamics 365 F&O to Power BI Reporting.

I am using Azure Synapse Link for Dataverse to fetch data from Dynamics, which will be plugged into migrated pipelines in Azure Synapse, which will take care of the rest.

This is one of the few deviations from original project, which used older version (Export to Data Lake in finance and operations), deprecated by Microsoft, where referenced were specific tables in F&O.

Problem I have is substantial, with solution nowhere to be found in the internet: I seem to be unable to replicate that granularity - when setting up the link in Power Apps panel all I get is this: enter image description here

But for my purpose I need this: enter image description here

For now the only way to get to picture 2 is to set up ASLD to use Spark pool: enter image description here

Is it truly the case? Is there no other way to be able to query tables from F&O, and not Dataverse entities?

That would mean extra cost of the Spark pool, which - as per Microsoft warnings about overhead of ASLD - may need more than one standard node and for much more time than with non-Spark link...

And since I have not much experience with Spark in Azure Synapse Analytics, I am not sure how difficult it will be to swap sources for the whole migrated solution. Any tips, please?


Solution

  • On the first screen of the New Link page:

    Do NOT check the box for Connect to your Azure Synapse Analytics workspace.

    On the next page:

    • Under Add Tables
      • Expand Advanced
        • Enable Show advanced configuration settings
        • Enable Enable Incremental Update Folder Structure

    Poof, magic. D365 Finance & Operations will appear. We never found these steps clearly spelled out in the documentation. After a day of dead ends, it was random clicks in the correct order that found the solution.