Search code examples
amazon-web-servicesapigoogle-cloud-platformclouddata-processing

Developing a simple cloud data processing system


What would be the most simple way to take a public data API, for example, and schedule a daily job to calculate a set of statistics and land the computed statistics in a cloud database?


Solution

  • What about using CloudWatch Event rule with Schedule Expression and a target of a Lambda function?

    The event rule would trigger your Lambda function, e.g., once a day. The function would then call the API, process the data from the API, and write the results into a DynamoDB or RDS database, depending whether you require relational or non-SQL database.