Build s3 Datalake Using Dynamo DB data source

Question

i'am a data engineer using AWS, we want to build a data pipeline in order to visualise our Dynmaodb data on QuickSigth, as u know, it's not possible de connect directly dynamo to Quick...u have to pass by S3.

S3 Will be our datalake, the issue is that the date updates frequently (for exemple column named can change / costumer status can evolve..)

So i'am looking for a batch solution in order to always get the lastest data from dynamo on my s3 datalake and visualise it in quicksigth.

Thank u

Answer 1

You can access your tables at DynamoDB, in the console, and export data to S3 under the Streams and Exports tab. This blog post from AWS explains just what you need.

You could also try this approach with Athena instead of S3.