Search code examples
azureazure-synapsepolybase

Can Azure Synapse query from external relational stores?


This diagram from this URL states Azure Synapse cannot query external relational stores but Azure databricks can.

But here I see it is possible with Azure Synapse. We could also use polybase in Azure Synapse. None of these articles are outdated. So what am I missing?

enter image description here


Solution

  • Your second URL is for External tables, which are not the same as an external relational stores (Azure SQL, MySQL, PostgreSQL, etc.) I do not believe any of the Synapse engines can connect directly to relational data stores [although I'm not certain of Spark's limitations in this regards], but Pipelines can. While they both use Spark, Databricks is a separate product and not related to Synapse.

    Polybase uses External Tables, which are metadata references over blobs in storage (Blob or ADLS). Synapse supports External tables in both Dedicated SQL Pool and Serverless SQL. Spark tables are also queryable from Serverless SQL because they are stored as Parquet files in ADLS. I believe this is also implemented as an External Table reference, although it does not display as such in the Workspace UI.