How to pull data from AWS Security Hub using Scheduler?

Question

How to pull data from AWS Security hub automatically using a scheduler ? I am new to AWS on doing some analysis I found below :

1. In Security Hub data is in Json format , we don't have option to do Export to csv/excel ?
2. All Security hub findings/insights are automatically sent to eventbridge ? Is it true ? If yes where i can check the same in eventbridge ?

Are there any other options in order to pull data from security hub , every 12 hours automatically. I want to take the data from security hub and pass it to the ETL Process in order to apply some logic on this data ?

Is Eventbridge the only and best approach for this ?

Answer 1

On:

1. It is a JSON based but it's their own format named AWS Security Finding Format (ASFF)
2. It is true (for all resources that SecurityHub supports and is able to see). It should be noted that Each Security Hub Findings - Imported event contains a single finding. In order to see those events you'll need to create an EventBridge rule based on the format for each type of event.

• how to create rule for automatically sent events (Security Hub Findings - Imported)
• In addition you can create a custom action in SecurityHub and then have an EventBridge event filter for it too.

Once you have that set up, the event could trigger an automatic action like:

• Invoking an AWS Lambda function
• Invoking the Amazon EC2 run command
• Relaying the event to Amazon Kinesis Data Streams
• Activating an AWS Step Functions state machine
• Notifying an Amazon SNS topic or an Amazon SQS queue
• Sending a finding to a third-party ticketing, chat, SIEM, or incident response and management tool.

In general, EventBridge is the way forward, but rather than using a scheduled based approach you'll need to resort to an event-based one. In order to intercept all findings, instead of rule being triggered by just specific one, you'll need to adjust the filter and essentially create a catch-all rule for SecurityHub which will then trigger your ETL job.

EDIT (as requested in comment):

The filter in the rule would look like this:

{
  "source": [
    "aws.securityhub"
  ]
}

with regard to the ETL, it really depends on your use case, having Kinesis Data Firehose dumping it to S3 and then using Athena as you suggest on your own would work. Another common approach is to send the data to ElasticSearch (or now OpenSearch). This blog post described them both, you can adjust it based on your needs.

EDIT 2:

Based on the discussion in the comments section if you really want to use a cron based approach you'll need to use the SDK based on your preferred language and create something around the GetFindings API that will poll for data from SecurityHub. You can use this function in Python, which extracts data from SecurityHub to Azure Sentinel as an example