Search code examples
devopsserverlessamazon-auroraamazon-quicksightaws-aurora-serverless

AWS Aurora Serverless (MySQL) with AWS QuickSight

So I've been trying to setup a BI tool with my Serverless backend i.e. Aurora MySQL.

Architecture as below:

enter image description here

Implementation on these lines: tutorial-link

Now, I'm exploring options for connecting this backend with a BI/Analytics tool for quick querying and visualisation purposes.

Since, our backend is a Aurora MySQL Serverless, the limitation would be to not be able to connect directly to the endpoint like how one could do with a provisioned one.

As stated here: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html

On similar lines, I'm trying to setup a AWS Direct Connect, which is currently in requested stage:

enter image description here

I need assistance whether this is right approach, if so, are there any tutorials for using Direct Connect with Aurora Serverless?

On the other hand, are there any bi tools anyone has successfully used with Serverless backends.

Need is to be able to get up and running in the shortest amount of time, to get quick inferences. And then move to ETLs or more complex use cases.

Any assistance much appreciated!

Thanks, Arnab

Solution

  • Figured this out.

    One needs to setup an Enterprise QuickSight Account to be able to connect to Private VPCs. The regular QuickSight Account expects the endpoints to be publicly available. While Serverless endpoints aren't.

    Limitations: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html

    Prerequisites: Enterprise QuickSight Account, Aurora Serverless cluster and make sure your cluster has active capacity units when creating the connection.

        Step 1: In QuickSight, choose your profile icon at the top right of the screen, then choose "Manage QuickSight". From the menu at left, choose "Manage VPC connections".

    Step 2: Choose "Add VPC connection" to add a new VPC connection.

    Step 3: For "VPC connection name", type a unique descriptive name. This name doesn't need to be an actual VPC ID or name.

    Step 4: Type the subnet ID for Subnet ID, and type the group ID for Security group ID. Make sure that the subnet and the security group are in the same VPC. Also, make sure you are accessing a VPC that is in the same AWS Region where you are creating Amazon QuickSight analyses. You can't use Amazon QuickSight in one AWS Region to connect to a subnet and security group that are in a different AWS Region.

    Important note: Please make sure the Security group you use for the VPC connection has access to the Security group used by your Aurora Serverless cluster and vice versa.

    Step 5: Review your choices, then choose "Create".

    Step 6: Create a new Aurora Data source

    Step 7: For the "Connection Type" option choose your newly created VPC connection from the drop down.

    Step 8 Fill out the rest of the details on for the connection and click "Create Data Source".

    You should now be able to select the table you wish to use in your analysis or write a custom SQL query to get the data.

    Maybe helps someone in future.