Search code examples
azure-sql-databaseazure-data-factory

Loading Data from Amazon Athena to Azure SQL using ADF


I was trying to load data from Amazon Athena to Azure SQL but I can't see the linked service. Anyone who know/have information how to load Data from Amazon Athena to Azure SQL using ADF will be appreciated.

There is no connector in ADF linked service.

How do we connect from Athena to AZURE SQL using ADF?


Solution

  • There is connector for Redshift and RDS but no connector for Amazon Athena unfortunately. One way to do it is to save your data into files in an S3 bucket then copy this using a copy data activity in ADF to copy to blob storage or where-ever you want in Azure. Then from there its easy to move it to your azure sql. You can of course use something else as intermediate step, but this seems to be the most straight-forward way. S3 connector is also one of the most used connectors, so if you run into trouble, it will be easy to solve.

    So your pipeline will look like this.

    | Amazon Athena -> Amazon S3 -> Azure Blob Storage -> Azure SQL Database |


    Here is your documentation for S3 connector